My 70 Gigabyte Mahler job

Today I completed my Gustav Mahler 11 Symphonies ripping job, requiring almost 70 GB free space from my ripping workstation. Not mentioning of two 3GHz CPU-s, running hot at 99% performance level.

Matherial

I have bunch of DVD-s in my posession. All Gustav Mahler's symphonies fom 1 to 10, plus Das Lied von der Erde. All DVD-s had:

• Video: MPEG-2 / 720x480 (NTSC) / 4:3;
• Audio: DTS audio + PCM stereo audio 48KHz 16bit

Original recording was done in stereo and DTS was artificially generated later.

What I did

I copied the contents of DVD-s to my workstation's HDD, using DVD Decrypter. Decrypting happened at the speed 7x - 10x, depending other simultaneous tasks. Then I converted original MPEG 2 video to much more efficient format MPEG 4 in AutoGK. It took about 1:1, eg 20 frames per second.

Result

I have now Gustav Mahlers 11 symphonies in MPEG 4 compression with hi quality PCM stereo sound.

Mpeg 4

Today I ripped The Matrix again and converted it into Mpeg 4 AVI (Xvid). My Matrix DVD is NTSC 16:9 format with english AC3 sound. In that case my particular workflow was:

DVD Decrypter ver 3.5.4.0 Read more
AutoGK ver 2.40 Read more and download

My main objective is to save some disk space, keeping high audio and picture quality. Also, my current media renderer D-link DSM-320 hangs playing huge VOB files. So, basicalyy I'm searching the way to playable.

First DVD Decrypter phase copies the raw content of DVD to HDD, removing copy protection. AutoGK phase converts number of 1 GB VOB files to one AVI file.

Size matters

VOB form: 5 GB (main movie + AC3 Dolby Digital 3+2/1 soundtrack only, 720 x 480)
AVI MPEG 4 1,86 GB (Target quality 100%, 720 x 304 resolution)

BitRate while playing: AVI 1954 Kbit/s; VOB 7600 Kbit/s (measured by DSM-320).

Viewing experience

First I looked video from my Dell Optiplex 20" Wide aspect monitor. I looked good.
Then I copied the AVI file to my Media Server and lloked it via D-Link DSM-320 media receiver on my Hitatchi 42" 1080HD plasma and Onkyo TX-L55 digital receiver. The result was disappointing. Sound was good -- as soundtrack is probably unaltered during compression, but image was unharp and had problems with moving objects and camera panning. Movements were jerky, not smooth. Soud was badly out of sync. Not recommended.

Next steps

Have to try other media renderers. May-be I have to build one, based on Intel SFF desktop PC?

Question

I have all Gustav Mahler symphonies in my posession, all DVD-s. If I'd like to rip only audio, what is the best method?

DVD has 2 soundtracks:

-> PCM 48 KHz, 16 bit
-> Dolby Digital 5.1

Quality is vital. I play music off mu server via UPnP media player D-Link DSM-320.

So, best technique? Please comment!

I found Mr Elliott's tutorial for conversion, going to try it.

Minor upgrade

Today I upgraded my Media Server RAID setup.

Before:
One volume per server

RocketRaid 1640 controller
RAID 5 volume, bootable (Win XP + data)

After

NEW: Two volumes per server
NEW: 40 GB IDE UDMA 5 bootable HDD (Win XP + TwonkyMedia server + other software, not shared)
NEW: RocketRaid 1810A controller (hardware accelerated RAID 5 parity calculation)
RAID 5 (4 x 160 SATA 150), containing digital music, images and movies - shared

So, basically I have bootable, not shared HDD for Win XP and data-only RAID 5 volume, shared to my house.

Performance IOmeter

Write throughput
Before 10 MB/s
After 54 MB/s Improvement 5,4x

Read throughput
Before 100 MB/s
After 100 MB/s same

Real-world copy test 254Mbyte CD

Before 8,5 MB/s
After 21,2 MB/s Improvement 2,5 times

Performance HD Tune

After (55MB/s, burst 82 MB/s) and before (47,7 MB/s, burst 57 MB/s)

I'm satisfied with new RAID 5 write performance!

Dirt cheap Russian option

You have to fill up Your huge media server with music. How? One option is iTunes, billing 1 € (U$ 1,2) per track. Another option is www.gomusic.com ,they bill just U$ 0,10-0,15 per track. Ten times less, one album for one track from iTunes.

Gomusic in the nutshell:

• > 31 000 albums
• > 25 000 artists
• fast download
• U$ 0,15 per track

It is mad to make U$ prepayment to russian internet company, selling online music? For most people it is. Still, this service has been reliable and fast. I have used it over 2 years. They claim,that legally is all correct with "Federation of Authors and Rightsholders for Collective Management of Copyright in Interactive" - so it's not pirate site.

DVD economics

How much is one DVD on Your digital home media server HDD in $$$? In VOB form? Lets take 5 GB Movie (The Matrix for example).

one 250 GB HDD = U$ 80.00
one Matrix = 5 GB
one HDD can home 50 movies

So, 1 movie occupies U$ 80/50= U$ 1,60 worth of hard drive.

In reality, You tend to have RAID, so multiply it by factor 1,25 (in 4 disk configuration)

1 Movie occupies U$ 2.00 worth of RAID5 disk system :) Not cheap, heh!

Half Terabyte server update

I downloaded best performance meter available -- Iometer. As the Iometer User's Guide says, Iometer is an I/O subsystem measurement and characterization tool for single and clustered systems. It was originally developed by the Intel Corporation and announced at the Intel Developers Forum (IDF) on February 17, 1998 - since then it got wide spread within the industry. Look more: www.iometer.org

So, I played in my home network, particularly with my HTMS (Half Terabyte Media Server) RAID subsystem.

My HTBMS server config:

• Dell Dimension 8200 base, 2GHz P4, 512MB RAM (tower)
• 4 PCI 33MHz x 32bit slots,PCI 2.2 compliance + AGP slot
• Intel PRO/1000 MT Gigabit NIC
• HighPoint RocketRAID 1640 SATA RAID conroller
• 4 x 160 GB SATA drives (WD + Seagate) in RAID 5 configuration
• MS Windows XP Pro
• TwonkyMedia UPnP media server

So, test data:
Maximum read throughput: 100 MB/s (with >1Mb packets)
Maximum write throughput: 10 MB/s
Maximum I/O ops/s: 10 000 ops/s (with 512 Kb packets)

I am not happy with write performance. Read performance is very good, also through gigabit NIC and my home lab, all gigabit. Problem for low write performance is parity calculation during write in RAID5 model. Option is to go for RAID 0 or 1, but I'd like to stay in RAID5, because of reliability and space. Buying model 1640 I didn't have any idea, that parity is calculated by computers CPU. So, processor load is 100% during large writes. I found that RocketRaid 1810a has hardware XOR parity module, so CPU is not overloaded. Have to swap RAID controller, or I'm not satified.

I found extremely interesting forum thread about buidling cheap and powerful RAID5. Must reading for everybody, who are trying it at home.

Geisha is Ripped

Today I sarted ripping DVD-s into my Half Terabyte Media Server (HTMS). Filling server with DVD-s is easy. Geisha was about 5 GBytes! I have 390 GB free, it can handle about 70 DVD-s only!!!! Shame...

I used DVD Decrypter for ripping. Later found from Wikipedia article, that this program is illegal. Is it so?

My ripping hardware is dell Optiplex 620 Dual Core 3GHz Intel Pentium D, with 2GB RAM. I ripped all DVD at 10x speed. So, 100 minute movie is in Your server in 10 minutes. Satisfactory!

Bad news is that olny first VOB file played, all the rest hanged my DSM-320. Have to play with options.

Good news is that TwonkyMedia server can handle huge VOB files, Windows Media Player 11 can't, too slow.

Wish List

This is Media Receiver wish list, concerning D-Link DSM-320, DSM-520 and upcoming DSM-750 models. I use model DSM-320 in conjunction with my home cinema. Soon I replace it with DSM-520 and old unit goes to bedroom, connected to old-school CRT TV. But I have wishlist for D-Link.

Please add LCD monitor D-sub output
Currently You may see the server contents, setup it and make adjustment only using your AV equipment -- LCD TV, Plasma TV; projector etc. DSM has composite, s-video and component outputs. I am lucky and have 42PD8600 42" 1080i Hitachi plasma TV. But I have a problem. If you listen all day long music and plasma is on, all static texts of DSM-320 "burn in". They disappear in some time. Still, You dont to keep Your 42" plasma on all day only to show track name currently playing. I miss some additional display. D-Links philosophy is not to add one or twil-line lcd to their receiverst, as some other companies do, like Roku. OK, I agree. But please give oportunity to add additional 15" PC LCD monitor via d-sub connector.

Screen saver

To avoit durnt in characters on my plasma screen

More resolution for OSD
Having about 260 artists and 700+ albums in my media server, I'd like to see a lot on the screen, trying to find album I wish to play. Currently You can see only 10 lines of text or so. TwonkyMedia has good solution indexing and grouping albums, artist etc to reduce time-consuming scrolling. Thats a good solution. But still, I'd like to see info in many colums, at least optionally.

Track properties
Please make possible to see bitrate, encoding and other properties of streaming tracks and also streaming video. Hope, that UPnP standards support it.

Thats it, three wishes only where is my goldfish?

Ripping in My Mind

Today I did some more ripping.

Ripping (also referred to as digital audio extraction) is the process of copying the audio or video data from one media form, such as Digital Versatile Disc (DVD) or Compact Disc (CD), to a hard disk (definition from Wikipedia).

My objective was to understand difference between WMA Variable and MP3 320 Kbit/s encodings. They both are my vavourite because they meet hi quality listening needs and still they both are compatible with most of equipment available.

MY test album was Denon's high fidelity demo CD, containing 11 tracks from Beethoven to Bruckner. Demanding stuff! Symphony finales and Piano contserts.

My surprise was quite big, then MP3 320 gave 23% bigger file than WMA Variable. I assumed, that encoder should choose very high bitrates because of nature of music.

• MP3 320 Kbp/s: 162 MBytes album size;
• WMA Variable: 132 MBytes album size.

Ichecked real compression levels of WMA Variable:

• Track 2 Ludwig van Beethoven, Egmont overture
  341 Kbit/s
  
• Track 4 Ludwig van Beethoven, Symphonie nr 7, Presto
  325 Kbit/s
  
• Track 10 Anton Bruckner, Symphonie Nr 6, Finale
  317 Kbit/s

So, no real differene in compression, WMA variabl is just more effective than MP3. I didn't hear any difference in sound, listening tracks after each other. I know, that A-B-X blind test is more objective, but I dont have equipment at my home for that.

I didn't make any fundamental conclusion. I keep ripping using WMA Variable, as I used to do earlier.

New fast WiFi @ home

Some people heard already about new products, like D-Link Dual-band draft-N router and DSM-750 media receiver, having so-called draft-N wifi. What is it? How fast is it?

WiFi evolution:

1999 - 802.11 b - 11 Mbit/s, typically 6,5 Mbit/s

2003 - 802.11 g - 54 Mbit/s, typically 23 Mbit/s

2007 - 802.11 n - 540 Mbit/s, typically 200 Mbit/s

Look also: D-Link DSM-750 media receiver

Look also: D-Link Dual-band draft-N router

Today I link my media receiver to my media server via wired LAN to avoid problems in video streaming. Current WiFi speed is not enough to guarantee bandwith needed for DVD-quality or HD video, if base station is some 10 meters away. Lets hope, that draft-N aka 802.11n does the job.

D-Link's new media receiver

Dlink announced some new features of upcoming draft 802.11n high definition media receiver DSM-750.

Evolution of DSM media players -- my interpretation:

DSM-320

Wireless 802.11g;

Screen resolution 480 lines? (not confirmed)

SCART RGB or Component + S-video

New in DSM-520 (compared to DSM-320)

HD screen resolution 1080 for MPEG 2 and MPEG4, 760 for WMA HD and 760 x 480 for photos;

Wireless 802.11b & g;

HDMI output

New in DSM-750 (compared to DSM-520)

Draft 802.11n wireless (up to 200 Mbit)

So, only remarkable upgrade is draft N wireless, what allows better video streaming over wireless network.

My 0,03 cents

What is the best and most rational compression method, ripping Your CD-s? To understand it better, I ripped one jazz album, containing 9 tracks of hi quality jazz tunes. I ripped it in 5 diccerent modes, using only good quality setting and skipping crappy ones. My Windows Media Player 11 compressions were:

WMA Loseless (about 250 MB) Not supported by most DMR's!

WMA Variable (about 90 MB) Ennuste recommends!

MP3 320 (about 90 MB) Ennuste recommends!

WMA Pro (about 55 MB) Not supported by most DMR's!

WMA 192 (about 55 MB)

MP3 192 (about 55 MB)

Loseless WMA produces about 5 times bigger file than 192 kbit/s compression. Widespread media receivers don't play Loseless. Usually they are limited to MP3 320 kbit per second rate.

My 0.02 cents

Use WMA Variable or MP3 320 kbit/s compression. You may not forgive yourself in the future, that compressed the music too economically. Lets bare in mind that original CD has 1411 mbit/s bitrate. You lose 66% disk space, using higher bitrate, but result transparent.

There is big quality difference between encoders, not all are created equal in this world. Crappy encoders show high bitrates, but result is not transparent. Use only well-known quality tools!

Economics

One CD album occupies about 90 MByte of hard disk, using hi quality 320 kbit/s encoding. What is the cost of this space?

WD Caviar SE16 250GB Hard Drive (Serial ATA II, 7,200RPM, 16MB - MPN: WD2500KS)
Price Range: $71.99 - $131.00, typically around $80.00. One HD can hold 2500 CD albums, so:

Cost of ONE CD on the hard disk = 0,03 cents

Media Server Software

I started experimenting with server softwares first time when I bought my first DMR (digital media receiver) from D-Link. The model was wireless media player MediaLounge DSM-320:
http://www.dlink.com/products/?sec=1&pid=318
and server software D-Link MediaServer was included in the box. Software is freely downloadable brom D-Link website. I tested MediaServer version 1.07. I don't describe througly this product. Instead, I write how I found software meeting my needs.

Media Server Evolution

As I told, I started with D-Link Media Server. In the nutshell my server software evolution is following:

D-Link Media Server -> MS Media Connect 2.0 -> MS Windows Media Player 11 Beta -> MS Windows Media Player 11 -> TwonkyMedia 4.1

D-Link Media Server - relatively slow in screen operations and music vas interrupted randomly for 0.5 - 1 sec using my old Celeron server;

MS Media Connect 2.0 - very same problems as D-Link;

MS Media Server 11 - It uses Media Connect 3.0 version, but no actual development in speed or usability. For the comment, that Vista uses Media Connect 4.0 version, but I haven't tested it yet.

Read more about Media Connect: http://en.wikipedia.org/wiki/Windows_Media_Connect

And the Winner is

TwonkyMedia ver 4.1 is clear winner, despite its a tad complicated web-interface setup. It has several very good usability features, hanling large music collections, lika Arist Index, Album Index etc. Plus, it has excellent technology, it stresses servers CPU minimally, uses cache-s to speed up recurring queries. Result is user-friendly functionality and very good speed moving around huge collections.

Main characteristics of the MediaServer

• Easy to setup,


• support for major media formats,


• convenient navigation on large media collections,


• very short response time,


• multiple language support,


• highly customizable (including navigation structure),


• easy handling of thousands of media items,


• Internet radio,


• interfaces to 3rd party applications like iTunes, Winamp, Adobe Photoshop Album,


• search actions allow you to easily find a specific piece of content (depending on client support)


• supports a large variety of UPnP streaming clients (see our device compatibility list)


• integrated client adaptation layer to support special media player capabilities,


• available for all operating systems and embedded devices like NAS, STBs and PVRs*,


• small footprint and very small memory consumption,


• stable code base deployed in various products and tested at UPnP and DLNA plugfests

Look also: http://www.twonkyvision.de/

Half Terabyte Media Server

"Half Terabyte" media server at home -- sounds good, yeah?! True. It was just few years ago then Microsoft demonstrated 1 TB storage SQL test. It was something! Now I am honored to own 0,5 TB storage at my home. Being exact, its not 0,5 TB, but 0,45 TB, but "Half Tera" sounds much nicer.

Base of my media server is Dell Dimension 8200, equipped with 2MHz Intel P4 and 0,5GB RAM. My son used it about 3 years as gaming and internet surfing entertainment equipment. It had 1 good feature -- big tower case, lots of internal HDD bays, 4 available.

I took out powerful sound card. I replaced Nvidia gaming 3D graphics card with passive (fanless) one. I throw out HDD.

Disk configuration

I installed 4 SATA disks 'a 160 GB into RAID 5 configuration. As my old base computer had only 32 bit PCI slots, I had tu use relatively slow PCI SATA RAID controller RocketRaid 1640 from HighPoint Tech USA.

RocketRAID 1640 Highlights
http://www.highpoint-tech.com/USA/rr1640.htm#top

• PCI 32bit@33/66MHz



• 4 Channel PCI to SATA RAID Controller



• Support up to 4 SATA Hard Drives(compatible with SATA II Hard Drives)



• RAID 0, 1, 5, 10 and JBOD



• Support Windows, Winx64, Windows Vista, Linux, FreeBSD

HD Tune gives some 47,4 MB/s average data transfer speed and 16ms acess to this system. Enough for home system, but a tad less what I was expected.

Network adapter

I replaced old CNet 100 Mbit NIC with Intel PRO/1000 GT card.
http://www.intel.com/network/connectivity/products/pro1000gt_desktop_adapter.htm

It gave some 60% raw throughput speed (yes, not 10-fold). Its ment for desktop computers and stresses CPU a lot. I should try server NIC and see, is it stressing less. Currently CPU is near 90% stressed during backup copies.

I installed MS Windows XP Pro SP2 with all updates. I installed TwonkyMedia, copied all data from my old server, shared directories for local users -- and voilà! My Half Tera server was up and serving!

I digitized it all (almost)

One day I made quite fundamental decision. No stacks of CD-s anymore in my house! I should rip them on the hard disk and listen from more innovative systems -- like digital media receivers all over my house.

I made quite MS friendly choice to rip them into WMA format: 192 bps, 44 Khz sampling, 2 channel, 16 bit format. For some reason I believed its sound superiority over mp3 format.

Also, I moved to digital photography -- that's indeed pretty obvious. I also started to scan all my previous films and photos into my digital storage.

Today I have 8 700 tracks from 680 albums, fron 230 different artists in my Media server, about 35 GB of songs.

Key facts of Digital Music:

• 35 000 000 000 bytes of music
• 8 700 tracks
• 680 ripped or purchased online CD albums
• 230 artists

Also I have 21 000 digital photos in my Media server, allocating 23 GB of disk storage. They occupy about 15% of my half-terabyte Media server. So, there is room for growth.

I don't have videos and digitized DVD-s yet in my Media server, as I use to rent films and look digital TV.

More about my Media server hardware and software, also about media receivers in my future blogs.

The First Post

Hello world! This is my first post. Even more, this is my first blog ever.

Exactly about what I'm going to post here? I will write about the things wherw I am good. Thewse thing are: photography; digital home (digital media receivers, servers, etc); cars (Porsches mainly). Yes, I know, it sounds childish from 42 year man with 2 children. But men are kids!

More about my gizmos, gadgets and solutions in the next blogs. All of them will be illustrated with photos, so keep following!

Looking forward....