HTPC

Getting SickBeard to generate pyTivo Metadata

Sickbeard is one of the best companion tools for anyone who uses SABnzbd+. However, by default, Sickbeard only creates metadata for XBMC (from what I could tell). I decided to get my hands dirty with a little python hacking to see if I could get it to create metadata that is used by pyTivo when sending media to TiVo devices. I'm pleased to announce that my hackery works most of the time. It has bugger-all error checking and will break with non-standard characters and other fun things, because I'm still getting my head around how python mishandles unicode.

Anyway, if you want to play with something that is almost certainly going to break the first time you try it, feel free to grab my fork from GitHub: http://github.com/Djelibeybi/Sick-Beard.

Kudos and thanks go to the pyTivoMetaThis project at SourceForge for the inspiration and not an insignificant chuck of code.

37 Comments

  1. Slappula says:

    This is a great idea, but I can’t seem to get it to work. I’ve copied the source files over those from the latest windows installer. It looks like the tv.py is the main script where you’ve made changes, but a lot of these changes are commented out. Does it generate metadata that will work with PyTivo’s Push function (or just Pull)? Any help would be greatly appreciated!

  2. djelibeybi says:

    The stuff that is commented out is just for the show metadata (which TiVo doesn’t actually use). The good stuff is in the episode metadata, way down at the end of tv.py. I don’t use pyTivo’s push function, so I can’t comment on that, but it should work the same way. You’ll need to paste the logs from Logs/sickbeard.log from when it attempts to write out the metadata for me to try and work out why it’s not working. However, there are a bunch of files in lib/ that are required to support the date mangling, so you need to make sure you have all of that too.

    Also, I run SickBeard exclusively on MacOS X and Linux, so I have no idea if it’ll work the same way on Windows.

    Though, I did say it was almost certain to break the first time you tried it, so at least I got that right.

  3. Slappula says:

    Thanks for the quick response dje. It was actually user error on my part. I reinstalled Sickbeard following the instructions on their site for a Windows install (http://sickbeard.com/install.html). I pointed the GIT tool to your fork instead of the one in the instructions. I’m running this on Windows Home Server. The Windows version of SickBeard asks you to use the sabtosickbeard.exe as the post-processing script for SABnzbd. It must be calling the out to the same Python scripts as the Unix/OSX versions- because it works perfectly! It looks like it actually builds the metadata for every show in the season folder which is awesome.

    It works with pyTivo push, but it doesn’t carry over the season and show number. It think this is a limitation of using push. Using the pull method (downloading from Tivo) may fix this, but I haven’t tried it yet. The cool thing about using push is that it groups them together in a folder. As long as you push them over in the proper order, the lack of season/show # shouldn’t be an issue.

    Thank you so much for creating this!!! Sickbeard, SABnzbd & your fork is an incredibly powerful combo. Now I just need an Automator-like tool on my WHS to copy the files to my push folder.

  4. djelibeybi says:

    Pull brings down all the metadata and groups in folders as well (uses the Series ID for this). Pull also orders them correctly, assuming that TheTVDB.com has the right Original Air Date. I’ve noticed some bad source data that sometimes screws up the order, so you’d do well do double-check on some of the less popular shows you may be watching.

  5. brennok says:

    I just wanted to say thanks for this. I need to rescan my library now so I can generate all the data. I also as mentioned on the forums posted an issue requesting this be implemented into the official version going forward.

    http://code.google.com/p/sickbeard/issues/detail?id=795

    • djelibeybi says:

      Thanks. Note that I’ve spotted a bug when zap2it has no Series ID for a show. They all get lumped together on the TiVo under the name of the first show that has “None” as the Series ID. :) I’m going to look into sorting that out sometime this week.

  6. brennok says:

    I haven’t played much with the pytivo metadata generator, but I believe you need to create a default txt file for each folder for ones without series id. You may want to post it on the pytivo forums and they may have an answer.

  7. djelibeybi says:

    Actually, I’ve already fix it (check Github for the latest version). Essentially, some shows don’t have a zap2it_id which is what it was using for the series ID by default. I’ve added a check so that if there is no zap2it_id value, it falls back to using the TheTVDB.com’s API Series ID for the show instead.

  8. brennok says:

    Ok downloaded and trying now.

  9. brennok says:

    Not sure if it is coming from the pytivo implementation, but I got this message this morning. I will have to check the logs later.

    2010-09-27 08:45:40.756000 POSTPROCESSER :: Exception generated in thread POSTPROCESSER: ‘NoneType’ object has no attribute ‘read’

  10. djelibeybi says:

    Yeah, that could well be my code. :) Check Logs/sickbeard.log to see if you find any actual Python errors. If so, post those here, please.

  11. brennok says:

    I will have to wait to see if it happens again. I forgot to check when I got back home and my log no longer shows it. I must have rebooted for an update or something.

    Also looks like the source was updated. I almost wonder if a separate automatic tool would be better for now until the implementation is added. I don’t know how difficult it would be though, but just something to monitor the tv folder and generate the new txt file when it sees a program.

  12. djelibeybi says:

    I’ve just updated my fork with the contents from the upsteam, so you can update if you like. If you want to use a separate tool, take a look at pyTivoMetaThis, which is what my code is based on anyway.

  13. brennok says:

    Thanks I never had much luck with pyTiVoMetaThis, but it has been a while since I tried. Last time I think I tried I kept getting just basic errors.

    Thanks for updating yours though.

  14. djelibeybi says:

    Yeah, pyTivoMetaThis works for me (on Linux) but I’ve never tried it on Windows. :)

  15. brennok says:

    Got this error tonight

    Oct-02 00:09:13 ERROR POSTPROCESSER :: Exception generated in thread POSTPROCESSER: unknown string format
    Oct-02 00:09:13 DEBUG POSTPROCESSER :: Traceback (most recent call last):
    File “C:\Sickbeard\Sick-Beard\sickbeard\scheduler.py”, line 54, in runAction
    self.action.run()
    File “C:\Sickbeard\Sick-Beard\sickbeard\autoPostProcesser.py”, line 49, in run
    processTV.processDir(sickbeard.TV_DOWNLOAD_DIR)
    File “C:\Sickbeard\Sick-Beard\sickbeard\processTV.py”, line 175, in processDir
    returnStr += processDir(ek.ek(os.path.join, dirName, curFolder), recurse=True)
    File “C:\Sickbeard\Sick-Beard\sickbeard\processTV.py”, line 175, in processDir
    returnStr += processDir(ek.ek(os.path.join, dirName, curFolder), recurse=True)
    File “C:\Sickbeard\Sick-Beard\sickbeard\processTV.py”, line 187, in processDir
    result = processFile(movedFilePath, dirName, nzbName)
    File “C:\Sickbeard\Sick-Beard\sickbeard\processTV.py”, line 575, in processFile
    rootEp.createMetaFiles()
    File “C:\Sickbeard\Sick-Beard\sickbeard\tv.py”, line 1271, in createMetaFiles
    result = self.createNFOs(epsToWrite, force)
    File “C:\Sickbeard\Sick-Beard\sickbeard\tv.py”, line 1418, in createNFOs
    text = parser.parse(myEp[“firstaired”] + ‘ ‘ + myShow[“airs_time”]).strftime(“%Y-%m-%dT%H:%M:%SZ”)
    File “C:\Sickbeard\Sick-Beard\lib\dateutil\parser.py”, line 697, in parse
    return DEFAULTPARSER.parse(timestr, **kwargs)
    File “C:\Sickbeard\Sick-Beard\lib\dateutil\parser.py”, line 303, in parse
    raise ValueError, “unknown string format”
    ValueError: unknown string format

    (Error trimmed)

  16. djelibeybi says:

    Yup, that’s my code: TheTVDB.com’s date values are dodgy at best, so something unexpected must be being returned in the XML. I obviously need a little more error checking. None of my shows have this issue, but that just means I’m slightly luckier than you. :)

  17. brennok says:

    Or it just means I am less picky about what I record or have on backup which is definitely the case lol.

    if you can would you delete the comment with the log after you copy it for obvious reasons ;)

  18. brennok says:

    Did you see the error I posted under the issue? I wasn’t sure if you were tracking that thread or not and if it was just the same as the one here. It also looks like he rolled out several updates.

    I also noticed on some transfers to the TiVo under the show description it would say nullfirst airdate or something similar with the actual date. I will have to find what show it did it on since not every show did it.

  19. brennok says:

    I hate to bother you, but are you planning on keeping up with this until he eventually merges the two? I would love to keep using it but as he is back from vacation and applying fixes I would prefer to keep up with the main version also.

    If not do you know what files I need to copy from one to the other just so I can keep my library showing the same and then just use your version to generate the pytivo metadata in the meantime?

  20. djelibeybi says:

    I suspect he’ll never merge the two (nor have I requested it) as the way I’m doing the metadata is sub-optimal. It would be better if he created a metadata plugin system, at which point I can just write the TiVo bits.

    I have been traveling for work recently, so I haven’t had a lot of time to keep this up-to-date. I’ll probably look at it again this weekend. I can’t tell you which files to copy from one to another, because I don’t know if he’s edited files that I’ve changed as well.

  21. djelibeybi says:

    But, I’ve just pushed the latest update to my fork, because it was only a single change to a single file. If you git pull from my repo, it’d be the latest version with my changes merged in.

  22. brennok says:

    Ok cool thanks. I was just meaning the files like the database file so that both versions see the most recent listing of shows. It looks like I just need to sync the DB copy between the two.

    I could see him adding it as an option eventually since not everyone would need it. He accepted the enhancement idea at least.

    Thanks for working on this though. I know only a few of us use it.

  23. brennok says:

    It looks like he is willing to look into at least based off his comment today. Thanks again for your work.

  24. brennok says:

    Hey sorry I think I pushed to the tree by accident. I was trying to get it to update on another computer and now I get an error when I try to pull. It is now not letting me update so I don’t know if I just messed something up local or if it went to your actual fork.

  25. djelibeybi says:

    My fork is not writable, nor does it list any updates, so I suspect you may have just messed something up locally. :)

  26. brennok says:

    good to know lol.

  27. brennok says:

    Thanks for the update. I was just curious if you ever sent midgetspy a pull request to review your code to see if he could add it as an option.

    http://code.google.com/p/sickbeard/issues/detail?id=795

  28. djelibeybi says:

    I’ve left a comment there, but I haven’t sent a pull request because the code is pretty crappy. :)

    I have now sent a pull request though.

  29. brennok says:

    Ok cool. Hey thanks again for working on this. I know it is just for personal use, but I appreciate you putting it out there since it isn’t anything I could ever pull off.

  30. djelibeybi says:

    NOTE: I’ve deleted this fork and will start working on a new fork based off the new metadata engine. Hopefully that is more likely to hit mainline, but I’m still not promising anything. :)

  31. brennok says:

    Works for me. I appreciate you working on it.

  32. brennok says:

    Might be something worth looking at adding if possible. http://tivocommunity.com/tivo-vb/showthread.php?t=375589

    It would be nice if the series ID matched the ones on the TiVo and this way they would group with the same shows if there were other episodes recorded already in Now Playing/My Shows.

    I haven’t tested it see if it works, but it looks like reneg has a list of show series IDs that match tvdb. I don’t know if any of the links even still work though.

  33. djelibeybi says:

    Yeah, I might not actually do any more for this because I’ve just replaced my TiVo with a few XBMC machines. I’ll still take a look, but don’t rely on anything.

  34. brennok says:

    Ok cool. I completely understand.

    I tried the HTPC route and couldn’t get into it. I just found after using the TiVos for so long playback doesn’t feel the way I prefer if that makes sense. I think it is just the delay that seems to happen on streaming versus recorded locally.

  35. djelibeybi says:

    Heh, I have the reverse: the XBMC boxes on my gigabit network playback instantly, while I have to keep waiting for the pyTivo transcoding to get the episodes onto the TiVo. With XBMC I can watch HD stuff instantly.

  36. brennok says:

    I am running the Premieres so the faster transfers come in handy. I deal mostly in SD for sickbeard since it is just things I am going to watch once or backups in case I miss a recording due to a conflict. As a result they are instant transfers and no transcoding that I can see with the speed. If I was dealing with HD content that wasn’t recorded by TiVo, I would probably have a HTPC or XBMC box running though I doubt I could give up on my TiVos.

    For me it is more the FF and REW. It just feels smoother and more accurate on the TiVo. I think I just never got the timing down on 7MC and since my TiVos are lifetime I put off upgrading the HTPC with a good tuner card.

Leave a Comment