Downsizing High Definition - HTDV to DVD

I haven't given up yet. I'm currently testing different methods of resizing. My HDTV capture is a 1080i broadcast with a 16x9 aspect ratio. So the resolution is 1920 by 1080, kind of. You see the "i" at the end of 1080 meanst that the program is interlaced. So instead of a true 1920 by 1080 broadcast you get 2 half height pictures per frame. In the first field (half frame) you get all of the odd lines, then in the second you get even lines. This method is also called BFF or "bottom frame first."

Simple resizing doesn't work because applications treat the half frames as a single frames. Resizing will introduce artifacts. There are two methods of resizing these interlaced pictures. The first is merging or deinterlacing the picture, then resizing the image, the reinterlacing it again. Remember this is a video source not film. Film is interlaced by the 3:2 pulldown method, but I digress. This method is slow and produces a temporary double frame, and doubles the frame rate, these large double frames have to be resized. On my 1.8GHz machine the frames are processed only 1.5 a second. For a video containing hundreds of thousands of frames this can take a very long time.

The second method is much faster but it has problems too. You can separate the fields, resize the half frames and then weave them back together. This can produce a flicker to the video, it can also introduce a problem called "Spatial Misalignment," this is where due to resizing factors the top and bottom frame do not match up exactly.

Why just can't it be simple? I'll report back when i find a compromise between quality and speed.

Audio Sync Problems Eliminated, Finally

Well, my suspicions were correct. The problem with the audio sync with my HD capture lies somewhere between ProjectX and Mpeg2schnitt. ProjextX has the ability of creating Mpeg2schnitt IDD files which are index file that contain metadata about audio/video synchonization. For some reason this file is ingnored or incorrectly used by Mpeg2schnitt during the editing process.

I loaded the elementary streams I demuxed with DGIndex into a AVS script. The resulting file information, when loaded in to VirtualDubMod, shows the video at 3 hrs 34 minutes, and the audio at 3 hrs 36 minutes. That kind of shows me that there are dropped video frames in the movie. The issue with using elementary streams, is that there is not metadata to detect and skip these dropped frames.

I demuxed again with ProjectX and this time used Cutteraman to edit the file, I'll cover this process in depth later. I indexed again with DGindex to create a D2V file. The resulting file, when loaded into VDubMod in an AVS script, shows the audio and video exactly the same length. Hot Damn! I think it's finally fixed. I'm going to mux and test it out now. Here's the script I used to test the video sync.

#Begin AVS script
LoadPlugin("c:\program files\dgmpgdec\dgdecode.dll")
LoadPlugin("c:\program files\avisynth 2.5\plugins\nicaudio.dll")
video=Mpeg2Source("k:\captured\brickyard 400\brickyard_edited.d2v")
audio=NicAC3Source("k:\captured\brickyard 400\brickyard_edited.ac3")
AudioDub(video,audio)
#eof

BTW, I'm re-encoding the audio during this process. I was originally just correcting the audio using BeSpliced, I may still do this I'm not sure. Some of the AC3 frames were of the incorrect size. Maybe I should fix it, then re-encode it... More on that later.

I'm maintaining a list of software I'm discussing on this post.

Audio Sync Problems Revisited

Now it looks like that I've repaired my source PVA file to the point that it is usable. I actually muxed it into a transport stream with ProjectX, I then ran it through Mpeg2Repair, which found only a few errors. I think the real sync issues are coming from Mpeg2Schnitt. That's unfortunate because I really like the interface over Cutteraman or VideoRedo (which I only have a trial for.)

I am demuxing the TS file with DGIndex, part of DGMpgDec, at the advice on someone from Doom9's forums. I'm going to attempt to edit out the commercials and remux the video. There's not a whole lot of software out their made to mux Mpeg2 program streams from elementary streams. Most are geared toward DVD authoring so they don't want the 1920x1080 source. I would like to be able to test the edited file for sync issues before I spend 24 hours re-encoding it to 720x480 anamphorphic Mpeg2.

Major Sync Problems Encountered

Well my Adventures in HDTV Capture story is on hold for a little while. I'm currently trying some different tools to correct the PVA files I've captured. Unfortunately video captured from terrestrial sources is supseptable to sync issues. When I demuxed my PVA's I ended up with an MPEG-2 video M2v and AC3 audio track that were of different lengths.

Apparently I'm not the only one with these problems. It seems that it has something to do with missing packets in the stream. When these files are demuxed with DGIndex the missing packets are simply ignored. Over the course of my 3 1/2 hour video capture the audio file is allmost a full minute longer than the video. I'm trying to capture and encode a Nascar Race so a small difference isn't really noticable. The post-race interviews, however, look like something out of an old KungFu movie.

I originally tried to run the files through ProjectX with some different setting to no avail. Then I attempted to run my PVA files through Mpeg2repair, which was originally written for Transport Stream, TS, files but should be able to handle and correct these files, but it doesn't seem to recognize the video stream. I then downloaded a commercial application called VideoRedo. The purchase price is $50 which is a little steep for my budget, especially since some of the most useful video and audio apps are free under the GNU GPL. Here's a quick list

• GordianKnot - DVD to AVI utility
• AviSynth2 - Frameserving and Preprocessing
• DGMpgDec - Mpeg2 decoder, indexer, deinterlacer, frameserver.
• VirtualDubMod - VFW video processor
  
• BeSweet - Audio apps by DSPGuru
• CDex - CD ripping utility
• Juice Receiver - Podcast downloading
• FFDshow - VFW codecs, slow development but there are newer builds lurking around the web. Official Build - Celtic Druid Build - Experimental Builds
  
• Audacity - Linear audio editing software. Awesome for making mp3 ringtones
• Xiph.org - Audio and video codecs like Ogg Vorbis and Theora, some are BSD licensed
  
• Rarewares - collection of audio and video encoders and codecs including Lame Mp3

But I digress. I'm also going to try the method of fixing the ac3 audio track with the method described on this thread in Doom 9's forum. Hopefully I'll be able to find some method to fix these errors without springing $50 for a VideoRedo license.

Adventures in HDTV Capture: Part 3 -Capture

I'm using DVBViewer TS (Technisat) Edition that came with my HDTV Tuner card. There is also a full edition but that costs money and I'm still in college. The lite edition that I'm using will only decode the MPEG2 video stream if I have the Sonic Cineplayer Mpeg2 Decoder Pack that came on my driver installation disc. The full edition does allow you to use the Mpeg2 decoder of your choice. At $19 I guess it's not a bad buy if the lite edition doesn't support your card or you don't have the Cineplayer decoder and already have another MPEG 2 Decoder. Nero, Windvd, Cyberlink Power DVD and Nvidia all offer MPEG2 decoders. FFDshow is an open source decoder but not very efficient for real-time decoding such as TV watching.
[Screenshot Coming]
After scanning for and setting up my channels. I tuned to NBC where the Nascar Race for the Chase are broadcasts. I used Antenna Web and Titan TV to ensure that I had all available channels entered. I also had to manually edit in some channels to receive them, but these were small independent stations and generally too weak to get a good signal. Keep in mind that DVB-T channels can run multiple streams. The NBC affiliate station in Memphis also broadcasts a music video channel and a local weather channel, all on channel 51.

A few other configuration issues I needed to change before I could capture. I had to edit my recording setting to select the option to capture to PVA. This saves the the file with the ".pva" extension. Depending on your capture program you should also be able to change this setting.

technical: PVA is a double acronym that stands for PES Video/Audio while PES stands for Packetised elementary stream. It's just a bit for bit copy of your video with no header or trailer.

[Screenshot Coming]

Hard Drive Storage:
I selected my second hard drive, a 250 GB SATA drive formatted with NTFS. Keep in mind that you need about 1 GB to record 10 minutes of HD broadcasting at 1080i. I should be able to keep about 40 hours of video on this drive, but that's just for storage. I also need room to demux, separate the audio from the video, so now its 2 GB for HD video, and once I have the commercials edited out I need another 2/3 of a GB. I also need room on this drive to save the encoded video and audio and remux the final project and author my DVD. So my rough calculations tell me that I can only record about 10 hours of video to this drive, assuming I don't delete the output of any intermediate steps and I'm not about to delete any of my steps until I'm absolutely sure I haven't made any mistakes, and my final product looks professionally recorded.

I changed the file split size to 4096 MB, That's not really necessary on an NTFS partition, but I wanted to be sure that if I needed to resync the video to the audio that I would have some start and endpoints to manipulate. YMMV. If you have any input on the subject I'd be glad to hear it.

That's all the setup we need, when your show starts just hit the record button and nothing else needs to be done. DVBViewer will automatically record your show to the drive you selected and you will find the PVA files there after you click Stop. DVDViewer also has a electronic programming and time recording feature that you can check out. But manual recording is all I need.

Next time I'll discuss how I demuxed my files and checked them for errors.

Adventures in HDTV Capture: Part 2

Ok, so I'm all set up to capture and I've read the forums and guides at Doom 9 and I've determined the steps I need to take to capture, edit and encode my video.

In this post I'll describe these steps and list the software that I'm going to use to accomplish these tasks. I'll probably keep this post updated to reflect other software I've discovered I needed so if you're following this adventure you might need to refer back to this post. Go ahead and bookmark it now. As I post each step I'll edit the list below to link to the post.

Steps

1. Capture - Record the video
2. Demux - Separate the Audio from the Video
3. Cut Commercials - TV shows are about 1/3 commercials
4. Process Audio - repair the audio track and correct the delay
5. Index the file for frame serving to the encoder
6. Avisynth2 Script Writing - for resizing and cropping
7. Encode Video - for transfer to DVD or DivX/XviD
8. Mux Audio Track - Putting it all back together

I recommend you only install the software as needed. Installing dozens of codecs, codec-packs, or video filters will only complicate the process. I recommend free open source software unless it is significantly inferior to commercial alternatives.

Software Needed

• DVBViewer or other DVB Capturing Software
• Java Runtime Environment - Free to use - for ProjectX and Bitrate Calculator
• MPEG2Repair - Free to use non-commercially - Repairs your captured files
• ProjectX - Open Source - Demuxes Transport Streams
• Mpeg2Schnitt - Open Source - Cuts out commercials, edits mpeg2 elementary streams
  
• Cutteraman - Open Source - Cuts out commercials, edits mpeg2 elementary streams
• BeSliced and Besplit - Corrects errors in AC3 audio files
• Nero AAC Encoder - Free - Processes audio for Mpeg4 files
• BeSweet and Besweet GUI - Open Source - Awesome audio encoding software
• Helix YUY Codecs v1.2 - Licensing Unknown - Colorspace Conversion
• DGIndex - Open Source - Demuxes and Indexes Mpeg2 files, also deinterlaces
• AviSynth 2.5 - Open Source - Preprocesses video and frameserves to the encoder
• VirtualDubMod - Open Source - Video For Windows (VFW) linear video processor
  
• MEGUI and x264 or XviD- Open Source - Creating downloadable Mpeg4 video
• QuEnc - Open Source - Encodes video for DVD
• Other Avisynth Filters - Mostly Open Source, all free - as needed for preprocessing video

Note that most of this software is experimental but all of it has worked for me. Most don't come with automatic installers either. I've created a folder called "DVB Software " and extracted the zip files to this directory. Avisynth filters need to be extracted to your AviSynth installation directory. In the next post I'll cover setting up DVBViewer and capturing the video and the settings that worked, and didn't work for me.

Read Part 3

Adventures in HDTV Capture: Part 1

When I bought my PCI HDTV tuner card I was stunned that I could get a device to capture and watch HDTV over-the-air broadcasts for less than $50. That's amazing because at the same time my dad was pricing set-top tuners for around $200. I purchased an Technisat Air2PC PCI tuner card sold by Broadband Technologies, that is capable of receiving ATSC HDTV signals. ATSC is the North American standard for HD Terrestrial broadcast, DVB-T. This is not the kind of card that will receive digital satellite, DVB-S, or digital cable broadcasts.

After installing the card and the DVBViewer TE (Technisat Edition). I still had problems viewing any signal although the software reporting I had an adequate signal from the card attached to a set-top amplified antenna. I email tech support and the kind folks at Broadband Technologies told my that I also needed to install the Sonic Mpeg2 decoder software that was on my installation disks. Silly me, I read through dozens of pages of documentation and the documentation never said that this software was necessary. I already have like half a dozen codecs installed that are capable of decoding MPEG-2 signals but DVBViewer needed another one.

Once Sonic was installed I got a perfect HDTV picture on my monitor. Now I don't have an HD Widescreen monitor but it is a 21" behemoth that I have set at a 1600 x 1200 resolution so that's good enough now. 1080i Broadcasts are 1920 x 1080 because my my monitor is only 1600 pixels on the horizontal axis the vertical resolution is scaled down. Now that I'm up and running I can just hit record and go. Not exactly. My next blog posts will cover my adventure of trying to capture and encode a 3 1/2 hour Nascar race. Not anything I would recommend now for a first timer. Hopefully through my exploits you'll learn how to do this yourself. Or what not to do.

Read Part 2

UT Bubba Sited in Memphis! Go Home REDNECKS!

For years now us Memphians have jokingly commented about our neighbors to the east being rednecks and hillbillies. Expecially those glorious idiots known as volunteers. "Hey let's go drink some shine and sit on uncomfortable metal bleachers along with 90,000 other drunken rednecks to watch a school we never would have been admitted play football." Now the evidence is clear, Memphis fans weren't simply judging people by the color of their necks. Go Tigers! 

SpamWars Episode I: Attack of the Drones

"Si Vis Pacem Para Bellum," If you seek peace, prepare for war. - Roman Motto.

When I was at home this weekend preparing for exams and doing yard work I had no idea that I was about to enter a war over a little Firefox extension called BlueFrog. It's an anti-spam utility that attempts to actively and aggressively fight spam, that is UCE, unsolicited commercial email.

I installed the extension and within 48 hours I received an email from a spammer, addressing me personally and threatening me to unregister from BlueSecurity's service or face an insurmountable amount of spam. Now I don't know if it is my military background, my sense of pride, or common sense that makes me naturally resistant to blackmail, coercion, threats, and extortion, but I would rather deal with 1000 spam emails a day than comply with these assholes.

It gets worse. I started visiting BlueSecurity's forums in order to get help on my issue and see if others had gotten similar emails. They had. But the users weren't the only concern BlueSecurity had today. They were under a DDoS attack from drone computers under control of the spammers. These are probably the same computers that are sending me unintelligble gibberish emails as I speak. As of 23:00 CDT BlueSecurity is completely offline. I managed to CoralCache their site before it tanked under the pressure, but right now that link isn't working either. Here's the BlueSecurity Communtity site CoralCache. Hopefully it's working by now.

This is about to get good. Here is the threating email verbatim:

hesperus cellular
1 message

---

Efrain Mcdermott	Mon, May 1, 2006 at 5:30 PM
To: EMAILADDRESSREMOVED@memphis.edu
Hey, You are recieving this email because you are a member of BlueSecurity (http://www.bluesecurity.com). You signed up because you were expecting to recieve a lesser amount of spam, unfortunately, due to the tactics used by BlueSecurity, you will end up recieving this message, or other nonsensical spams 20-40 times more than you would normally. How do you make it stop? Simple, in 48 hours, and every 48 hours thereafter, we will run our current list of BlueSecurity subscribers through BlueSecurity's database, if you arent there.. you wont get this again. We have devised a method to retrieve your address from their database, so by signing up and remaining a BlueSecurity user not only are you opening yourself up for this, you are also potentially verifying your email address through them to even more spammers, and will end up getting up even more spam as an end-result. By signing up for bluesecurity, you are doing the exact opposite of what you want, so delete your account, and you will stop recieving this. Why are we doing this? Its simple, we dont want to, but BlueSecurity is forcing us. We would much rather not waste our resources and send you these useless mails. Its simple, we dont want to, but BlueSecurity is forcing us. We would much rather not waste our resources and send you these useless mails, but do not believe for one second that we will stop this tirade of emails if you choose to stay with BlueSecurity. Just remember one thing when you read this, we didnt do this to you, BlueSecurity did. If BlueSecurity decides to play fair, we will do the same. Just remove yourself from BlueSecurity, and make it easier on you. Efrain Mcdermott I think maybe he was saying "Let me the hell out of here!When he let the coverlet fall into a smoking heap at the baseboard, there was a big smoking bald spot in the middle of the wall, but the paper was out."Colter,he said.What would she think, he wondered, of that man as he looked now, forty pounds lighter and ten years older, his legs a pair of crooked useless horrors?.