Home

The wizard is largely aimed at recovering video files from corrupted or deleted memory chips that were formatted with FAT32.  The process involves scanning the complete device so will slow if a large drive is analysed this way.  The wizard works for most files with typical extensions of .MP4, .3GP, .MOV  (although .MOV covers several different formats).

3GP and MP4 files are often not stored sequentially on a disk and so special tools have to be developed to recover the files.  This wizard is designed to detect, analyse and recover as many files as possible.  This can include reconstructing missing elements of the video file, in particular the moov atom which is essential to viewing the video.

The wizard is primarily designed for memory chips, but will also work on hard disk drives.  Files on hard disk drives will typically be sequential, while deleted files on a FAT32 chip are typically not sequential.  The wizard aims to detect the file type and create a playable video.  For both media types, attempts will be made to reconstruct videos that are not complete.

The main way to operate is via a Raw Scan -which scans the complete disk as described below.  An option under development is for Scan and Save files which will concentrate on just repairing damaged files, before doing a scan of the unallocated area of the disk.

The wizard works in a few stages

Stage one:

The complete disk is scanned for elements of video files.  Typically it will be looking for MP4 tags for the main atoms, such as 'ftyp' 'mdat' 'moov' and 'free'.  each location is then saved in an internal table

Stage two:

A moov element is looked for and if a sequential complete one is found, details are parsed.  This indicates the type of camera, type of file, and sometimes the structure expected on the disk

Stage three:

At this point, possible files are constructed starting with a 'ftyp' atom, and then looking (typically) sequentially for the 'mdat' and 'moov' atoms

Stage four:

The file is parsed to see if valid.  If it is not valid then full recovery takes place.  At first attempts are made to tie the mdat and moov atoms together.  If this fails, attempts will be made to reconstruct a moov atom from scratch. This last stage does require a valid moov atom to be present on the disk.

Thumbnail

When the wizard makes a good recovery a thumbnail image will be displayed - even on the demo. For the demo ut gives a high level of confidence that the video will be fully recovered.  (Not available on Windows XP systems)

Warning messages

In the scanning process certain inconstancies are sometimes detected, and a ***WARNING*** message displayed

***WARNING ***  A free atom found larger than  cluster size

  It is possible the cluster size is incorrect - Press F1 for more help

  Try cluster size of 0x40

This error indicates that the cluster size is not consistent.  A FREE atom is normally smaller than a cluster, and used to pad to the end of a cluster.  If the video has been moved from it's original memory chip, then there may a different cluster size.  This would also be true if a memory chip was copied to an NTFS hard drive.  In this case, the warning message can be ignored.  If on the original memory chip, the 'Try cluster size of 0x??'  is a guide value to be tried.

***WARNING ***  ftyp atoms have been found not on cluster boundary

  It is possible the cluster size, or start sector is incorrect.  Press F1 for more details

  Try start sector of 0x4

  And/Or try smaller cluster size

This error indicates a possible problem with the start sector value.  The ftyp atom is normally only found at the start of a cluster.  Any other location indicates a possible problem.  As in the previous warning, copying files to a hard drive can cause false positives.

Some of the file/camera types currently supported.  Many other cameras will match these variations

Camera

       Canon EOS700

       Canon SX600 HS

       Canon HF-G30

GoPro Hero - Black and Silver, all variations including high and low resolution files

Samsung HMX-H300

Kodak Zx1 Pocket Video Camera

Panasonic DHC

Sony PWM-F3

Video format (codec)

AVC1

MP4V

JPEG - under develoment

Logical file layouts supported

The following link is one that will grow on a regular basis until all common combinations are fully supported.  The level of recovery will vary on each type but will eventually include repairing the following types of failure

• Finding each cluster from a fragmented file (often deleted on FAT32 disks)
• Creating a moov atom when it is missing
• Creating a ftyp atom when missing - this in effect will allow raw mdat data to be displayed

The aim for each disk type is to make the video playable even when areas are missing.

MPEG recovery

This function also looks for and combines MPEG fragments.  It is intended to be used on hard drives and will attempt to find runs of MPEGs and then join them in to sequences.  The matching is not always perfect, so sequences should be checked for false matches.

Cluster size issues

Sometimes warning message will displayed indicating that the cluster size, or start sector may be incorrect.  The routines expect a FTYP atom to be at the start of a cluster, ie byte offset 4.  It also expects that if there is a FREE atom it normally only goes to the end of the current cluster.  If these conditions are not met, a warning message is displayed.

The default cluster size and start sector are generated from the file system information, but is not always correct.  This can be very true if images have been moved to a different type of drive.

The solution is not always obvious, and may be a combination of cluster size and start block.  However, for a modern memory chip, the cluster start is normally 0x0 and the cluster size for a FAT32 is very often 0x40.  It may be necessary to try a few variations

Repair not implemented

At times a message such as "Repair not implemented M4_FTYP_FREE_MOOV_FREE_MDAT" will be displayed.  This means that this possible repair or reconstruction mode has not been implemented yet.  Contact CnW if it is required.  The intention is to cover all permutations, but development time is not unlimited.

File type not recognised

At times the program may not recognised the structure of the data and will request to send a diagnostic log file to CnW.  The file is mp4_scan_<date>.$$$   eg mp4_scan_20130528.$$$.  This stores the locations of each ftyp, mdat etc found on the disk scan and will help CnW analyse the memory structure.  There is no user data in the file, just sizes and locations.  NB, because the file is stored in the CnW Temp directory, it is cleared down each time the program starts.

Scan range

When processing camera memory chips generally speaking it is normal to scan the complete chip.  Hence the options for start of scan, and % to scan will 0 and 100.  If dealing with a large hard drive, the scan may be very slow.  For a 3TB drive we are looking at many hours just for the base scan.  For this reason, it is possible to select just an area of the disk to scan, maybe the first 5% as in the example in the screen image above.

Type of memory device

There are two types of file system that can be recovered from, the original camera memory, and a logical copy, typically a hard drive.  Each will expect data stored in a different way.  For camera memory, it is often for the basic sections to be stored out of sequence, and at times the moov atom may not have been generated.  For a logical drive, the file is normally in sequence, but may be fragmented as the result of copying files to a previously used disk drive.  Both of these approaches require slightly different recovery processes,  by setting the option to either camera memory chip, or hard drive the best results may be obtained.

• Camera memory chip - when processing the chip that was in the camera, ie will be physically out of sequence
• Hard drive - when processing files that have copied to another device - ie will be physically sequential

Video type

The default setting is Auto.  For memory chips this is probably the best setting.  For hard drives there is a significant chance that the drive will contain my than one type of video.  The option of video type will allow focusing on the relevant type of video file.  The problem (for the user) is to establish which type of file is required.  The only safe way is to look at a known good file and hence determine the correct type.  For more details look at M4 disk layouts or the list below.  For hard drive, this option allows the selection of specific file types.  Thus if a Canon camera is to be recovered, the correct file type can be selected.

Save 2GB Image

This option will save the first 2GB of the memory chip image, to the selected output directory.  This works, even on the demo version.  The purpose for this option is to help solve any problems. The 2GB file can be sent to CnW Recovery (possibly using the free program www.wetransfer.com).  The file name will be called  2g_image_1234abcd.img   where 1234abcd is the Machine ID code.

Development Status

The Wizard function one of many elements.  Only certain stages have been completed and the table below shows the status for each type of video.  There are three basic processes, and they are tried in turn.  If a video can be verified, nothing more is don, if it fails verification, then the next stage is processed

• Stage 1 - Recover.  Assume that the data is in the normal sequence for the format
• Stage 2 - Repair.  Assume that the basic sequence is correct, but the moov or mdat has become fragmented.  ie find all the pries and put them back
• Stage 3 - reconstruct.  This is when typically the MOOV atom is missing (or too badly corrupted).  Data that looks like video data is parsed, and the MOOV atom is reconstructed.  For this to work, there must be at least one valid video on the memory device

* represents a format typically found on hard drive, rather than on raw memory chip

The above list does change on a regular basis.  For some of the repair and reconstruction it is dependent on the type of video codec used

File type and possible camera

Camera memory chip

M4_MDAT_FTYP_MOOV_FREE

       Canon EOS /Rebel range, eg 600D, 70D

M4_FTYP_MOOV_FREE_MDAT

       GoPro Hero 3 Black

       Ambarella

       M4_FTYP_MDAT_MOOV

               GoPro Hero 4 Silver

               Nikon D5100

       Panasonic DMC-GH4

       Canon SX100 HS

Hard drive formats

       M4_FTYP_MOOV_FREE_MDAT

               Canon 700

       M4_FTYP_FREE_MOOV_FREE_MDAT

               Kodak Zx1 Pocket Video Camera