YouTube URLs - Object reference not set to an instance of an object.

May 26, 2012 at 11:10 PM

Hello.

Sorry if this was already reported but I could not find it.

Does the program has support for YouTube URLs?

When the program tried to download 1080p trailer "I Still Have A Soul  (Trailer Compilation)" from a YouTube URL I had this error:

"26/05/2012 18:54:23 - Next trailer: I Still Have A Soul  (Trailer Compilation)
26/05/2012 18:54:23 -
    480p       http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=35
    720p       http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=22
    1080p      http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=37
    poster     http://static.hd-trailers.net/images/i-still-have-a-soul-post:er-20120525.jpg

26/05/2012 18:54:25 - 1. Unhandled Exception....
26/05/2012 18:54:25 - Exception: Object reference not set to an instance of an object."

And then the program finishes. Using version 2.07 and feed "http://feeds.hd-trailers.net/hd-trailers".

Then I manually included the trailer in the exclusion list to allow me to continue.

Thank you ijourneaux for software.

Regards!

Coordinator
May 27, 2012 at 8:29 PM

Interesting. Unfortuantely I can't support youtube trailers as I don't get a file size from the YouTube download link.  That said, you souldn't get the Unhandled Excpetion error.

On my machine, when I encounter the trailer you have, I get

5/27/2012 2:18:11 PM - Next trailer: I Still Have A Soul  (Trailer Compilation)
5/27/2012 2:18:11 PM - 
    480p       http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=35
    720p       http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=22
    1080p      http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=37
    poster     http://static.hd-trailers.net/images/i-still-have-a-soul-poster-20120525.jpg

5/27/2012 2:18:15 PM - Extracted download url (1080p): http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=37
5/27/2012 2:18:15 PM - Local directory: C:\Users\Ian\Videos\HD-Trailers.Net Downloader\2012-05-27 I Still Have A Soul  (Trailer Compilation)
5/27/2012 2:18:15 PM - Filename : I Still Have A Soul  (Trailer Compilation)-trailer.mov
5/27/2012 2:18:15 PM - Stream successfully opened
5/27/2012 2:18:15 PM - Invalid trailer size (-1 bytes) from Server. Skipping ...

Could you upload you config file to pastebin so I can take a look at it?

Coordinator
May 27, 2012 at 10:43 PM

I think I may have found what the problem is. Your config file may point to the problem.

Coordinator
May 28, 2012 at 1:38 PM

Figured it out.  In your config file, you have the flag set to create a movie NFO file. In this case, ther is no IMDB entry that matches the Title as this is a Trailer Compilation. I am trying to figure out how best to handle this. In this case, the trailer won't download anyway as it is from Yahoo but in the general case, if it is a compilation, should I download even if I can't create an NFO?

May 28, 2012 at 3:23 PM
Hi

thank you. Right now I'm not at home to confirm if I configured to generate NFO, but anyway you seem to be sure of that. The funny thing is that I don't know why I configured this because I don't need NFO files. So at home I will just disable this.

Regarding your question. If the user configured to generate NFO, probably the user need it somehow (maybe the NFO will be used by XBMC or other tool). But even so, I think your tool should download the trailer(s) even if it couldn't create the NFO file. For 2 reasons:

  • we assume that the external tool that uses NFO files (if any) should be able to properly handle the lack of a NFO. Of course, this may lead to undesired behaviour (like for instance if the user configured XBMC to play trailers that match the genre of the movie, in such case XBMC might play a trailer that does not match or it might just ignore the trailer).
    Even so, the behaviour of what to do with a trailer that misses an NFO is not up to TrailerDownloader but instead it is up to the external tool that uses the NFO. In the worst case TrailerDownloader will waste time downloading a trailer that later will be ignored, but to me this is acceptable.
  • we also have cases, like mine, where the user configured the NFO generation just to have it there, without actually using it.
That's my point of view.

Regards and thanks for your work.

João

2012/5/28 ijourneaux <notifications@codeplex.com>

From: ijourneaux

Figured it out. In your config file, you have the flag set to create a movie NFO file. In this case, ther is no IMDB entry that matches the Title as this is a Trailer Compilation. I am trying to figure out how best to handle this. In this case, the trailer won't download anyway as it is from Yahoo but in the general case, if it is a compilation, should I download even if I can't create an NFO?

Read the full discussion online.

To add a post to this discussion, reply to this email (hdtrailersdler@discussions.codeplex.com)

To start a new discussion for this project, email hdtrailersdler@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe on CodePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at CodePlex.com


Coordinator
May 29, 2012 at 4:09 PM

Configuring HD-Trailer.Net Downloader not to create NFO files would only have prevented the crash you observed. In this case, it would not have downloaded the trailer in question as the trailer was from YouTube. Currently downloading trailers from YouTube is not supported as YouTube doesn't supply the number of bytes in the trailer.

I beleive I have fixed the problem with the NFO file. I don't have a solution yet on downloading YouTube Trailers.

Ian

May 29, 2012 at 5:12 PM

Hi, 

I hope that I'm posting in the right thread here... probably should have opened a new one - but hey... here you go: 
I fixed the youtube-download and now I am succesfully able to download youtube-trailers in 720p or 1080p with the downloader.

However - I make use of two external libraries (both of them open source) in order to do so:
YoutubeExtractor - taken from flagbug over here: https://github.com/flagbug/YoutubeExtractor 
JSON (which is being used by YoutubeExtractor): http://json.codeplex.com/

Using those two libs I created an additional class for the youtube-download which sorts everything out (like quality and format) and will return a single download-string which then can be used by the normal WebClient (as far as I can gather it relies upon a proxy to serve a static url.

For example: 
Looking for a trailer for "I Still Have A Soul" it finds an entry "I Still Have A Soul  (Trailer Compilation)" with youtube-links - the 720p extracted by the downloader would be http://www.youtube.com/watch?v=dCoclAd-VRs&fmt=22 (although the &fmt=** doesn't do anything any longer! - I remove it manually from the link since hd-trailers.net still posts them for whatever reason). The lib I use the returns the following URL: 
http://o-o.preferred.fra02s01.v1.lscache7.c.youtube.com/videoplayback?upn=fbPmJWrcCg0&sparams=cp,id,ip,ipbits,itag,ms,mt,ratebypass,source,upn,expire&fexp=914055,909902,907217,907335,921602,919306,919316,904455,912804,919324,912706,904452&key=yt1&mt=1338303611&ipbits=8&itag=22&sver=3&signature=4E692D84C7297F86AAC8DC372FC6CCDD272A774E.68CDAF8F8043841700DAAFDE121A8830BF1CF6E7&ratebypass=yes&source=youtube&expire=1338328321&ms=au&ip=193.0.0.0&cp=U0hSTVNNVl9HS0NOMl9LR1pEOkZGaTVaRXdGSHRK&id=742a1c94077e551b

 

That link is handable by the classes you use and it spits out the number of bytes, so no problems there. Tested this with a few trailers and it seems to be working fine. However - if you consider implementing this I could offer you my code which does this.

May 29, 2012 at 5:26 PM
Edited May 29, 2012 at 6:55 PM

On a side-note: I can do 480p as well - the only problem with that is, that the resulting file is a webm-file. I couldn't say if XBMC for example can handle it, VLC certainly can though.

On another note: When it downloads via YouTube there is no NFO written. I suspect it has something to do with the changes you made above? What exactly would I have to do to get that back working (don't really have the time to read through the whole code...:-) )

On a third note: You should consider if you implement this. As far as I can tell Youtube's TOS prohibit downloading any material, so it may be against their TOS to implement this.

Coordinator
May 30, 2012 at 11:41 AM

That is great. I would love to add that to the program. If you can share your code, I will try to implement over the next couple of weeks. (Traveling for the next several days).

The reason there is no NFO created for the trailer compilation "I Still have a Soul" is that it can't find an entry for that on IMDB.  I suppose I should create an NFO that only has limited information.

Coordinator
May 30, 2012 at 11:46 AM

Regarding the YouTube TOS, here is the relevant statement

"You shall not download any Content unless you see a “download” or similar link displayed by YouTube on the Service for that Content."

So, it would appear that  if there is a download link, the video can be downloaded.

May 30, 2012 at 1:15 PM

Yes I had understood that. I was concerned with the crash.

Tks

Em 29/05/2012 12:09, "ijourneaux" <notifications@codeplex.com> escreveu:

From: ijourneaux

Configuring HD-Trailer.Net Downloader not to create NFO files would only have prevented the crash you observed. In this case, it would not have downloaded the trailer in question as the trailer was from YouTube. Currently downloading trailers from YouTube is not supported as YouTube doesn't supply the number of bytes in the trailer.

I beleive I have fixed the problem with the NFO file. I don't have a solution yet on downloading YouTube Trailers.

Ian

Read the full discussion online.

To add a post to this discussion, reply to this email (hdtrailersdler@discussions.codeplex.com)

To start a new discussion for this project, email hdtrailersdler@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe on CodePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at CodePlex.com

May 30, 2012 at 3:46 PM
Edited May 30, 2012 at 3:57 PM

Ah - okay that explains the missing NFO. Of course it would be missing - I missed out on "Trailer Compilation" being a trailer compilation of various movie-titles' trailers. I thought it was a compilation of the movie "I still have a soul" (which apparantly doesn't exist :D ).

However: 
The diff of the program.cs can be found over here:
http://tbueter.com/pastebin/view/raw/e66274e0
(I commented the unused variables out as reported by my compiler)

And the fetcher_YouTube.cs I created addtionally:
http://tbueter.com/pastebin/view/4d4ea1e0

I referenced two binary dlls taken from the libs mentioned above:
Newtonsoft.Json.dll (taken from this release: http://json.codeplex.com/releases/view/82120 )
YoutubeExtractor.dll (taken from here: https://github.com/flagbug/YoutubeExtractor/blob/master/Releases/0.1.0/YoutubeExtractor-0.1.0.zip?raw=true )  

Coordinator
May 30, 2012 at 9:56 PM

I made the same mistake with "I Still have a Soul". Thanks for the code I will try to incorporate as soon as possible.

Coordinator
May 30, 2012 at 9:58 PM
jgslima wrote:

Yes I had understood that. I was concerned with the crash.

Tks

 

I should ahve an update out dealing with this shortly.

Aug 17, 2012 at 5:32 AM

I've having a similar error, but it is with a yahoo URL, even though I have the config set to skip yahoo trailers. Normally it skips them with no problem.

Log:

17/08/2012 2:25:15 PM - Next trailer: Keep the Lights On  (Trailer)
17/08/2012 2:25:15 PM -
    480p       http://playlist.yahoo.com/makeplaylist.dll?sdm=web&amp;pt=rd&amp;sid=132556832
    poster     http://static.hd-trailers.net/images/keep-the-lights-on-1344960999-poster.jpg

17/08/2012 2:25:15 PM - 1. Unhandled Exception....
17/08/2012 2:25:15 PM - Exception: Object reference not set to an instance of an object.

Config:

<!--Set url parts to skip. Comma delimited. Ex: "Apple.com,IGN.com"-->
    <add key="SitesToSkip" value="yahoo.com,ign.com,hd-trailers.net,aol.com,youtube.com"/>

Coordinator
Aug 17, 2012 at 7:04 PM

Are you using the latest version? I will check this weekend.

Coordinator
Aug 17, 2012 at 10:51 PM

Doens't look like I will be able to replicate. All of the trailers download for me but the one you were having trouble with has scrolled off the feed.

 

Aug 18, 2012 at 2:40 AM

Running version 2.0.10, does version 2.0.11 include any extra fixes?

And yep, now that trailer is gone from the feed, it is working correctly again now.

Coordinator
Aug 18, 2012 at 8:07 PM

The latest version matches the source code that is uploaded. It does include a few small fixes but nothing major. As you noticed, I don't think the application pays attention to the sitestoskip key.  I haven't made any changes there since I took over the code so it may have been broken for a while.

Aug 19, 2012 at 1:47 AM

sitetoskip definitely works, which is good, otherwise so many duplicates are downloaded. Eg my notification emails often show the following:

Trailer source (yahoo.com) is identified as Site To Skip in config. Skipping...

Trailer source (yahoo.com) is identified as Site To Skip in config. Skipping...

Trailer source (yahoo.com) is identified as Site To Skip in config. Skipping...

Trailer source (yahoo.com) is identified as Site To Skip in config. Skipping...

Trailer source (yahoo.com) is identified as Site To Skip in config. Skipping...

Trailer source (yahoo.com) is identified as Site To Skip in config. Skipping...

Coordinator
Aug 19, 2012 at 3:57 PM

Interesting then that you had the exception error.The check for SitestoSkip should have occurred prior to any real work with the download lin that would ahve resulted in the exception. I will have to take a look at it. 

Feb 9, 2013 at 1:16 AM
Still get this issue every so often even though yahoo trailers are set to be skipped. Latest one is as follows:

9/02/2013 12:08:20 PM - Next trailer: lucía y el sexo (Theatrical Trailer)
9/02/2013 12:08:20 PM -
<img src="http://static.hd-trailers.net/images/error.png" width="16" height="16" style="border:0px;vertical-align:middle" alt="Yahoo! Redirection Script" title="Yahoo! Redirection Script" /> http://blog.hd-trailers.net/2012/12/23/yahoo-redirection-script/
480p       http://www.hd-trailers.net/yahoo-redir.php?id=0e090da7-2ac5-395d-84b5-ae54ae11a857&amp;resolution=480
poster     http://static.hd-trailers.net/images/sex-and-lucia-poster-20130207.jpg
9/02/2013 12:08:20 PM - 1. Unhandled Exception....
9/02/2013 12:08:20 PM - Exception: Object reference not set to an instance of an object.
Coordinator
Feb 19, 2013 at 11:48 PM
I think I found the problem. I have a regex search that is going off into never-never land. I have a work around and will upload a new version when I have a chance.
Feb 22, 2013 at 5:27 AM
Just to chime in, I'm experiencing similar issues:
2/21/2013 9:26:51 PM - Next trailer: Inescapable  (Clip: I Have Connections)
2/21/2013 9:26:51 PM -
    <img src="http://static.hd-trailers.net/images/error.png" width="16" height="16" style="border:0px;vertical-align:middle" alt="Y
ahoo! Redirection Script" title="Yahoo! Redirection Script" /> http://blog.hd-trailers.net/2012/12/23/yahoo-redirection-script/
    480p       http://www.hd-trailers.net/yahoo-redir.php?id=2321e5a8-527a-3bc6-b362-35166f0d31d7&amp;resolution=480
    poster     http://static.hd-trailers.net/images/inescapable-poster-jpg_230421.jpg

2/21/2013 9:26:51 PM - 1. Unhandled Exception....
2/21/2013 9:26:51 PM - Exception: Object reference not set to an instance of an object.
Coordinator
Feb 24, 2013 at 12:26 AM
Could you try version 2.0.13? When looking up the IMDB information, parsing the Rating and Year info was causing the regex to hang hard.
Apr 11, 2013 at 10:38 AM
Hi ijourneaux, thanks for the new version. I have installed 2.0.15, but unfortunately still getting the same issue sometimes. See below.

11/04/2013 7:33:00 PM - Next trailer: Caroline and Jackie (Theatrical Trailer)
11/04/2013 7:33:00 PM -
<img src="http://static.hd-trailers.net/images/error.png" width="16" height="16" style="border:0px;vertical-align:middle" alt="Yahoo! Redirection Script" title="Yahoo! Redirection Script" /> http://blog.hd-trailers.net/2012/12/23/yahoo-redirection-script/
480p       http://www.hd-trailers.net/yahoo-redir.php?id=fde64934-58c3-32f9-8ac2-43d20dd11c17&amp;resolution=480
poster     http://static.hd-trailers.net/images/carolineandjackie-poster-jpg_221001-resized.jpg
11/04/2013 7:33:00 PM - Looking up MovieName on IMDB
11/04/2013 7:33:03 PM - 1. Unhandled Exception....
11/04/2013 7:33:03 PM - Exception: Object reference not set to an instance of an object.
Coordinator
Apr 21, 2013 at 6:53 PM
I am trying to track this one down. Things are quite a bit more reliable now but Still having trouble with Yahoo downloads.
May 15, 2013 at 4:48 AM
Seems it isn't just yahoo trailers causing the problem (see below). Or is the IMDB lookup causing the issue?

14/05/2013 2:03:40 AM - Next trailer: The Butler (Trailer No. 1)
14/05/2013 2:03:40 AM -
480p       http://avideos.5min.com//753/5177753/517775245_2.mp4
poster     http://static.hd-trailers.net/images/the-butler-poster-20130507.jpg
14/05/2013 2:03:40 AM - Looking up MovieName on IMDB
14/05/2013 2:03:46 AM - 1. Unhandled Exception....
14/05/2013 2:03:46 AM - Exception: Object reference not set to an instance of an object.
Coordinator
May 15, 2013 at 1:12 PM
The stupid error is in the IMDB lookup. One of the regex lookups is running into trouble.

Unfortunately I haven't been able to trap the error before hand and handle it more gracefully. Will take a look at it tonight