pasemi
Info about the MPEG Movie feature, as found on some Sony digital video- or photo camera's, and some Samsung and Sanyo.
First of all: the quality of these Sony MPEG Movies is lousy!
I studied MPEG-1 compression and MPEG Movie EX / MPEG EX for a specific application: extend the usage of my Sony PHD-A55 Cyberframe (a digital Photo Frame) with non-upgradable firmware.
It can display JPEG (Exif 1.1), MPEG EX and even JPEG with MPEG EX layer-2 audio! For JPEG I already accomplished this extension. ANY JPEG! Now ANY videosource!
MPEG (Movie) EX
- Resolutions: 320X240 or 160x112 pixels
- With or without audio (MPEG-1, layer-2)
- No capture limit in time (until the Memory Stick is full)
History of Sony proprietary MPEG Movies:
MPEG-1: SQ(X), EX, HQ(X), VX
MPEG-2: AD
All these Sony proprietary MPEG-1 use the same logic, with a few differences. MPEG EX is MPEG-1 compliant and definitely NOT MJPEG or Motion-JPEG!
MPEG-1 works with Group Of Pictures (GOP) and at least I-, and probably P- and/or B- frame-types. I-frames (=Intra-encoded, the picture itself) alone are much like JPEG's. Motion-JPEG is a sequence of JPEG compressed files (only I-frames/JPEG's, no GOP's).
Sony may say MPEG EX has a frame rate of 8 frames per second (fps), but ... What I do think they mean is a 'capture rate' of 8 fps. The video is actually playing @25fps.
How is this possible? What I DISCOVERED is that Sony uses MPEG-1 in a smart/tricky way.
(Their chip/firmware designers and programmers should know).
MPEG EX has an 'IPP GOP' (now irrelevant: no B-frame), this means first an I-frame and then 2 P-frames. The P-frame only contains the (P=Predicted) differences compared to the I-frame or another P-frame. Thus consuming less bytes. In essence this is what MPEG compression is all about. P-frames work with overlay, motion estimation and motion compensation techniques.
3x8=24. MPEG EX has time instructions to display these frames @25fps (a little bit faster, but not noticable).
The applied P-frames in MPEG EX are extreme ones: no change when compared with the I-frame (zero motion estimation and zero motion compensation). They only reference the I-frame (instruction: frame 2 and 3: display yourself exactly like the I-frame from time x to time y). So dummy frames.
What actually happens: @8fps a picture is taken, compressed and a fixed byte/bit-string (not calculated, but already in their registers) is appended after that to accomodate the 2 dummy P-frames, calculated pointering (time-synchronisation) and after each 3 IPP GOP's a calculated stuffing/padding is used, to fill up to a physical block of 32Kb (320x240) or 8Kb (160x112). This block size takes into account the Memory Stick physics. After the movie has been captured, it can be displayed @25fps! The first frame is the original first(compressed) photo, the second and third frames are exact copies, the fourth frame is the second compressed photo etc. To check: there are tools that can show each MPEG frame by frame. When you zoom in, you can see NO difference at all between the first, second and third frame.
The older camera ((cheap) processor) is simply not fast enough to capture @25fps (and do complex MPEG compression in real-time). The bitrate is too high to cope with.
MPEG HQ(X) has an 'IIP GOP' and also displays @25fps. There are 2 I-frames and 1 P-frame (referencing the second I-frame). 2 I-frames are 2 captured and compressed photo's @16fps. Slightly different proces, but better quality, because of 2 different source photos per GOP. Whether the processor needs to be faster, I don't know, because I know there is time limited MPEG HQ(X) (15 seconds?). Perhaps there is better usage of memory.
MPEG VX is more like MPEG HQ(X): also IPP GOP, but resolution 640x480. Its physical block size (containing 3 GOP's) is 4x32Kb=128Kb I read the Sony DSC-F828 can capture 640x480 MPEG VX, @30fps with audio.
In my spare time I'm working on a way to make MPEG EX by software (there are no software tools yet that can output this specific file format). Input is a sequence of JPEG's. Most video-editing tools can output these ... Of course I would accomodate for (conversion to) the other formats as well.
This forum brought me a new idea: encode the produced sequence of JPEG's in NLE on computer to AVI. 'Print' the video from computer back to mini-DV (DV-in) and then convert in-camera to MPEG EX on Memory Stick (some Sony camcorder support this last step).
How did I find out all this? Simple: make some small footage, use some video-editing tools, a HEX-editor for comparison (find patterns), common sense, reference material, time, patience etc.
Here is an interesting old article (not mine) about the older SQ(X) format, in case you would for instance like to manually join MPEG Movie (remember; MPEG SQ(X) has lower resolution and probably another GOP):
http://www.mav-magazine.com/Aug1999/MPEG/
I'm know much more than is in the article (examples):
I have been able to exactly cut an I-frame manually, replace it with another (same resolution, exact cut) I-frame from another MPEG EX, fix 1 or 2 (relative) pointers and stuffing/padding (the other I-frame may be a little bit longer or smaller in bytes) and actually SEE the altered MPEG EX display on Sony hardware (I tested on PHD-A55 Cyberframe or my camera: DSC-S85) to prove my findings! The firmware accepted it!
I found the sequence header in the MPEG EX: the flag load_intra_quantizer_matrix equals 0, meaning NO QM! So the default QM specified by the MPEG standard is used (I know that QM).
First of all: the quality of these Sony MPEG Movies is lousy!
I studied MPEG-1 compression and MPEG Movie EX / MPEG EX for a specific application: extend the usage of my Sony PHD-A55 Cyberframe (a digital Photo Frame) with non-upgradable firmware.
It can display JPEG (Exif 1.1), MPEG EX and even JPEG with MPEG EX layer-2 audio! For JPEG I already accomplished this extension. ANY JPEG! Now ANY videosource!
MPEG (Movie) EX
- Resolutions: 320X240 or 160x112 pixels
- With or without audio (MPEG-1, layer-2)
- No capture limit in time (until the Memory Stick is full)
History of Sony proprietary MPEG Movies:
MPEG-1: SQ(X), EX, HQ(X), VX
MPEG-2: AD
All these Sony proprietary MPEG-1 use the same logic, with a few differences. MPEG EX is MPEG-1 compliant and definitely NOT MJPEG or Motion-JPEG!
MPEG-1 works with Group Of Pictures (GOP) and at least I-, and probably P- and/or B- frame-types. I-frames (=Intra-encoded, the picture itself) alone are much like JPEG's. Motion-JPEG is a sequence of JPEG compressed files (only I-frames/JPEG's, no GOP's).
Sony may say MPEG EX has a frame rate of 8 frames per second (fps), but ... What I do think they mean is a 'capture rate' of 8 fps. The video is actually playing @25fps.
How is this possible? What I DISCOVERED is that Sony uses MPEG-1 in a smart/tricky way.
(Their chip/firmware designers and programmers should know).
MPEG EX has an 'IPP GOP' (now irrelevant: no B-frame), this means first an I-frame and then 2 P-frames. The P-frame only contains the (P=Predicted) differences compared to the I-frame or another P-frame. Thus consuming less bytes. In essence this is what MPEG compression is all about. P-frames work with overlay, motion estimation and motion compensation techniques.
3x8=24. MPEG EX has time instructions to display these frames @25fps (a little bit faster, but not noticable).
The applied P-frames in MPEG EX are extreme ones: no change when compared with the I-frame (zero motion estimation and zero motion compensation). They only reference the I-frame (instruction: frame 2 and 3: display yourself exactly like the I-frame from time x to time y). So dummy frames.
What actually happens: @8fps a picture is taken, compressed and a fixed byte/bit-string (not calculated, but already in their registers) is appended after that to accomodate the 2 dummy P-frames, calculated pointering (time-synchronisation) and after each 3 IPP GOP's a calculated stuffing/padding is used, to fill up to a physical block of 32Kb (320x240) or 8Kb (160x112). This block size takes into account the Memory Stick physics. After the movie has been captured, it can be displayed @25fps! The first frame is the original first(compressed) photo, the second and third frames are exact copies, the fourth frame is the second compressed photo etc. To check: there are tools that can show each MPEG frame by frame. When you zoom in, you can see NO difference at all between the first, second and third frame.
The older camera ((cheap) processor) is simply not fast enough to capture @25fps (and do complex MPEG compression in real-time). The bitrate is too high to cope with.
MPEG HQ(X) has an 'IIP GOP' and also displays @25fps. There are 2 I-frames and 1 P-frame (referencing the second I-frame). 2 I-frames are 2 captured and compressed photo's @16fps. Slightly different proces, but better quality, because of 2 different source photos per GOP. Whether the processor needs to be faster, I don't know, because I know there is time limited MPEG HQ(X) (15 seconds?). Perhaps there is better usage of memory.
MPEG VX is more like MPEG HQ(X): also IPP GOP, but resolution 640x480. Its physical block size (containing 3 GOP's) is 4x32Kb=128Kb I read the Sony DSC-F828 can capture 640x480 MPEG VX, @30fps with audio.
In my spare time I'm working on a way to make MPEG EX by software (there are no software tools yet that can output this specific file format). Input is a sequence of JPEG's. Most video-editing tools can output these ... Of course I would accomodate for (conversion to) the other formats as well.
This forum brought me a new idea: encode the produced sequence of JPEG's in NLE on computer to AVI. 'Print' the video from computer back to mini-DV (DV-in) and then convert in-camera to MPEG EX on Memory Stick (some Sony camcorder support this last step).
How did I find out all this? Simple: make some small footage, use some video-editing tools, a HEX-editor for comparison (find patterns), common sense, reference material, time, patience etc.
Here is an interesting old article (not mine) about the older SQ(X) format, in case you would for instance like to manually join MPEG Movie (remember; MPEG SQ(X) has lower resolution and probably another GOP):
http://www.mav-magazine.com/Aug1999/MPEG/
I'm know much more than is in the article (examples):
I have been able to exactly cut an I-frame manually, replace it with another (same resolution, exact cut) I-frame from another MPEG EX, fix 1 or 2 (relative) pointers and stuffing/padding (the other I-frame may be a little bit longer or smaller in bytes) and actually SEE the altered MPEG EX display on Sony hardware (I tested on PHD-A55 Cyberframe or my camera: DSC-S85) to prove my findings! The firmware accepted it!
I found the sequence header in the MPEG EX: the flag load_intra_quantizer_matrix equals 0, meaning NO QM! So the default QM specified by the MPEG standard is used (I know that QM).