Magic files

Normally, to determine the file type, we use the file command. The way it works is that it uses a file /etc/magic to determine the type of file. Now if you have a custom file type, and you want to determine if a file is of that type, a magic file can be used with the file command to determine this. e.g. if the file you are working with is of the type school, (definitely not a standard type but you may want to work with it) a magic file may come in handy. Magic files usually have the extension .mgc . To use a magic file with the file command specify the -m flag e.g. file -m magicfile.mgc filetocheck .

Magic file structure

To create a magic file, it is necessary to understand the structure of the files it will be detecting. Since in Unix (and its successors) a file is merely a sequence of bytes, magic files use this sequence to determine what type the file is. man magic gives an exhaustive implementation of magic files but the basics are as follows. Each file starts with a test that each file will undergo. the sequence of the test is as follows:

• offset - gives the offset, in bytes, into the file in which the data is to be tested. Think of this like the position in the file from where the test starts.

• type - The type of data to be tested e.g. byte, float, string etc.

• test - the value to be compared with.

• message - the message to be printed if the comparison succeeds.

A test for the JPEG2000 file is:

0 string \x00\x00\x00\x0C\x6A\x50\x20\x20\x0D\x0A\x87\x0A JPEG 2000

This file would compare a file's contents at 0 offset and see if they contain the 12 characters specified, in hexadecimal code, and if they match prints JPEG 2000 on the screen.

To add further tests, for instance, in this case to tell the subtype, use the > sign.

i.e. at byte 20 check if it is jp2

>20 string \x6a\x70\x32\x20 Part 1 (JP2)

Adding mime-type

Additionally the mime type could be included with the magic file. A mime type declares the nature of the document. Whether it is a text file, or an image etc. See Mozilla's excellent explanation of MIME types for more information.

Mime types are added like this:

• !:mime - which shows the mime type is about to be declared.

• MIMETYPE - the actual type.

  For example the mime type for the jpeg 2000 jp2 is image/jp2.

Compiling it to a magic file.

After creating the file, compile it to a magic file as follows. file -C -m magicfilename. The compiled file can then be used with the file command as follows: file -i -m magicfilename filetobechecked .

Example

Create a magic file school.mgc that can be used with the command file to detect School data files. School data files always contain the string SCHOOL at offset 0.

0 string SCHOOL School data
!:mime School

I hope you have a nice time creating magic files for the win.