Wednesday, July 28, 2010

The Bong Loader

BongLoader.exe is a console application written in C#. I have developed and tested it on my windows system, but thanks to Mono it runs without any changes on my Linux Box. (DS209). 


What does it do:
  • it is executed via cron-job every 10 minutes.
  • read settings from config file (BongLoader.exe.config)
  • it checks if allready another instance of the applications is running. In this case the applications exits. (I don't want parallel downloads to minimize load on the NAS box).
  • It asks the database for the next waiting item.
  • If there is one it downloads the file from the Bong.TV server (http-get).
  • during the downloads the progress field in the database is updated (and also the FinishDate).
  • if the download was successfull the status is set to 'Finished'
  • if there was an error during the download, Status is set to 'Failed' and the error(text) is written into ErrorText-field.
  • the application writes to a log-file
  • applications exits

The Bong Parser

BongParser.exe is a console application written in C#. I have developed and tested it on my windows system, but thanks to Mono it runs without any changes on my Linux Box. (DS209). 

What does it do:
  • it is executed via cron-job every 30 minutes.
  • read settings from config file (BongParser.exe.config)
  • Read current recordings from Bong.TV server. It comes in the shape of a xml-file which has to be downloaded via http-get.
  • parse xml and create a list of recordings (List<Recordings> itemList)
The structure looks like this:

    class Recordings
    {
        public int Refnr { get; set; }
        public int BongId { get; set; }
        public string Title { get; set; }
        public string Description { get; set; }
        public string Channel { get; set; }
        public DateTime BroadcastDate { get; set; }
        public TimeSpan Duration { get; set; }
        public string ImageUrl { get; set; }
        public string Quality { get; set; }
        public string MediaUrl { get; set; }
        public string MediaFilename { get; set; }
        public int Status { get; set; }
        public int Deleted { get; set; }
        public DateTime StartDate { get; set; }
        public DateTime FinishDate { get; set; }
        public int Retries { get; set; }
        public string Category { get; set; }
        public int Progress { get; set; }
        public string ErrorText { get; set; }
    }
  • inserts each of the items of the list into the MySQL database (table content). It checks of course if the item allready exists in the database (checks this via BongId).
  • checks if there are hanging downloads. (This is done via the FinishDate. If Status = Downloading and FinishDate is more than 0.5 x Duration away from the current time, a hanging download is detected). In this case the entry is reset. (Status -> Waiting, Progress -> 0%, etc.)
  • checks if entries are successfully downloaded. In this case the recording is deleted from the Bong.TV server via http-delete.
  • a logfile is written (for each step)
  • applications exits



Tuesday, July 27, 2010

The Table Structure

In the moment there is only one table 'content' which looks like this:


The first 11 fields come directly from the BONG.TV rest-service for each recording.

I have added:

  • Status (which can be 0=Waiting, 1=Downloading, 2= Failed, 3= Finished)
  • Deleted (0=File still on the BONG.TV server, 1=Deleted from the Server)
  • StartDate (DateTime when the Download process stared)
  • FinishDate (during a transfer it is updated by the progress function and when a download finished it shows the finish-time)
  • Retries (if the download process failes, the status is set to 0 and so the download will start again after some time)
  • Category (the category for the video - with my self defined categories)
  • Progress (the download progress 0-100% during the transfer)
  • ErrorText ( the error reason if a download failed)

MySQL on Synology DS209

To use MySQL on the DS209 it must be enabled. It is actually preinstalled and must only be activated in the WebServices Tab of the DiskStation Manager.

I want to use MySQL and PHP (which i will introduce later) on my DS209, so the settings looks like this:



To manage the sql-database I use phpMyAdmin, which is not preinstalled, but is is possible to download a package with phpMyAdmin from the Synology Website, which can be easily installed on the DS209.

With the MySQL Admin tool i created a database called 'bong.tv' and a table 'content'. I also created a user 'remote' with all rights to the database.

BONG.TV Download Automation

OK, here is the overall structure of my Bong.TV download automation. First of all: All applications run completely on the DS209. There are actually 2 applications the ‘Bong-Parser’ and the ‘Bong-Downloader’. ‘Bong-Parser’ communicates with the Bong-TV Servers and gets an XML-list of all recorded videos and meta data like title, description, channel, download-URL etc. For each entry in the XML file, ‘Bong-Parser’ creates an entry in the content-table. Bong-Parser is started periodically, executes and closes. Of course it checks if an entry is already registered in the database. ‘Bong-loader’ now goes through the db-list and downloads the videos and stores them in the DS209 video directory. After a video is downloaded successfully, Bong-Parser deletes the files from the Bong-Server via a REST-Service. It takes also care that broken downloads are resumed.
In the next post i will describe the content table and the state-machine which takes care that everything runs smoothly.

.NET alias Mono on the DS209

I found out that I can install Mono on my Synology DS209. It has a 1,2 GHz Arm compatible processor and luckily Mono is included in the IPKG Packet system. As i’m a Windows developer i use putty to connect to the DS209 from my Windows machine. This might be interesting for other Diskstations: http://www.synology-wiki.de/index.php/Welchen_Prozessortyp_besitzt_mein_System%3F.
So i have made the following steps to install mono:
1. Connect with putty as root to the DS209
2. cd /volume1
3. wget http://ipkg.nslu2-linux.org/feeds/optware/cs08q1armel/cross/stable/syno-mvkw-bootstrap_1.2-7_arm.xsh
4. sh syno-mvkw-bootstrap_1.2-7_arm.xsh
5. ipkg update
6. ipkg install mono
7. Enter mono to check it runs.

BONG.TV Online Videorecorder

Since last month i’m a happy subscriber of bong.tv. (http://www.bong.tv) It’s an online video-recorder which allows me, to remotely record all the German speaking TV-channels (around 30 free receivable stations). The recordings are done in h.264 with full PAL-resolution of 720 x 576 pixels. Everything is stored ‘in the cloud’ on a 30 Gbyte sized storage, from where i can stream it directly (with my IPhone or PC- based browser) or download it. The latter is my favorite way to consume the videos as i additionally got a Sitecom-HD Player which plays the downloaded videos from my Synology DiskStation DS-209 where i have enough terabytes to store some material.
In the the following post i will describe, how i setup and develop a system which is able do the download and managing more or less automatically. The goal is to run the software completely on the DS-209 !