Odb File Format Version 1.9

In the current version, ODB stores all its data in a single file:

• Meta-model : classes that are stored in the object base
• Objects thats are stored
• Indexes

File Format Big Picture

ODB file format has the following block types:

• Header Block

• ID Block

• Class Info Block

• Object Block

Indexes are stored in the database as normal objects using the org.neodatis.odb.core.meta.ClassInfoIndex class. This class is marked as a system class (org.neodatis.odb.core.introspector.ClassIntrospector.getClassCategory(String fullClassName) to

Conventions :
1 Long = 8 bytes
1 Integer = 4 bytes

In the current version of ODB, entities are stored in the database file as two ways linked list.

Let's take an example to understand that:

An object which is managed by a NonNativeObjectInfo (Layer 2) object contains the data of the objects, the OID of the its class (ClassInfo) and some pointers:

• A pointer to the next object of the same type
• A pointer to the previous object of the same type

This pointers allow the navigation from an object to other. Like a two ways Linked list. This is simple way to handle object navigation but have some drawbacks for the Isolation property of ACID): See here the new ODB file format for version 2 to resolve them.

Database Header

ODB Header

This table is currently being edited! (06/04/2008)
Changes from 1.8 are in red

No	Field	Description	Position	Size
1	Use encryption	To indicate if database file use encryption	0	1 byte
2	Version	The ODB File format version	1	1 int
3	Original Language	A flag to keep the original language used to create database (Java=1,C#=2…)	5	1 byte
4	Database ID	A unique identifier of the ODB database file	6	4 Long
5	Is Replicated	To indicate if the database is being replicated	6+4*8=38	1 byte
6	Last Transaction sequential ID	The last Transaction Sequential ID	39	2 Long
7	Number of classes	The number of classes in meta model	39+2*8=55	1 long
8	First ClassInfo OID	The OID of the first ClassInfo of the meta model	63	1 long
9	Last ODB Close status	To check last close status : 1 is ok, 0 is not	71	1 boolean
10	Database String Encoding	The Database String Encoding, default is UTF8	72	50 bytes + 8 bytes for string size = 58 bytes
11	Is user/password protected	Does it need a user/password to access database: 1=yes,0=no	130	1 boolean
12	User name	The user name, if exist , if does not exist 'no-user'	131	50 bytes + 8 bytes for string size = 58 bytes
13	User password	The encrypted user password , if no password : 'no-password'	189	50 bytes + 8 bytes for string size = 58 bytes
14	Current block id position	Which is the current id block to be used	247	1 Long
15	First ID Block	The first block of ids	255	1000 bytes

Header size = 255
Total header size, including first id block of 1000 entries = 255 + 18034 = 18289

ID Block

ID Block

The ID Block is used to store all ids of objects and classes. In the ID block, the following data are stored:

• The Object/class ID
• The OID type (Object or class)
• The physical position of the object with the OID
• An indicator to say if OID is still active. Deleted objects are marks as deleted, for example.

The ODB header holds the position of the first ID block(field 10).

The ID Block can contains up to 1000 OIDs definitions. It has a fixed size :

ID Block size = size of header + 1000 * size of a repetition = 34 + 1000 * 18 = 18034 bytes

The ID Blocks are managed by the org.neodatis.odb.core.io.IdManager class. The actual writings in database file are done by the
org.neodatis.odb.core.io.**ObjectWriter **

> ID Block Header

No	Field	Description	Position	Size
1	Block size	The Block size	0	1 Int
2	Block type	20 (org.neodatis.odb.core.io.BlockTypes.BLOCK_TYPE_IDS)	4	1 byte
3	Block status	To indicate if block is full,1=no,2=full	5	1 byte
4	Prev block position	The previous ID block position	6	1 Long
5	Next block position	The next ID block position	14	1 Long
6	Block number	The block number	22	1 Int
7	Max id	Max id already allocated in this block	26	1 Long

> ID Block repetition

No	Field	Description	Size
1	ID type	To indicate if it is an ID of Object or Class	1 byte
2	OID	The Object ID	1 Long
3	Status	The status of the OID. 0=Unknown, 1=Active, 2=Deleted. See org.neodatis.odb.core.io.IDStatus	1 byte
4	Position	The phisical position of the object with this OID	1 Long

Class Info Block

Class Info block

The meta model of the database is stored using org.neodatis.odb.core.meta.ClassInfo which is a meta-representation of a native class.

> Class Info Header

No	Field	Description	Position	Size
1	Block size	The Block size	0	1 Int
2	Block type	4 (org.neodatis.odb.core.io.BlockTypes.BLOCK_TYPE_CLASS_HEADER)	4	1 byte
3	Class Type	System class:1, User class:2	5	1 byte
4	Class id	The id of the class	6	1 Long
5	Prev class info OID	The OID of the previous Class Info	14	1 Long
6	Next class info OID	The OID of the next Class Info	22	1 Long
7	Number of objects	The number of object of this class	30	1 Long
8	First object OID	The OID of the first object of this class	38	1 Long
9	Last Object OID	The OID of the last object of this class	46	1 Long
10	Full class name (with package/namespace)	The name of the class	54	Variable
11	Class attribute definition position	The position where to find the class attributes definition	-	1 Long

Writing implemented in org.neodatis.odb.core.io.ObjectWriter.writeClassInfoHeader(…)
Reading implemented in org.neodatis.odb.core.io.ObjectReader.readClassInfoHeader(…)

> Class Info attributes

Class Attributes are managed using org.neodatis.odb.core.meta.ClassAttributeInfo.

Attributes can be one the Object Info : Native Object Infos or Non Native Object Infos

if attribute is a Native Object

No	Field	Description	Position	Size
1	Is Native	1 if native	0	1 byte
2	ODB type id	1 (org.neodatis.odb.core.meta.ODBType)	1	1 int
3	Name	Attribute name	5	variable
4	Is used in index	If attribute is used in an index	-	1 byte

if attribute is a Native Object : array

No	Field	Description	Position	Size
1	Is Native	1 if native	0	1 byte
2	ODB type id	org.neodatis.odb.core.meta.ODBType	1	1 int
3	Sub Type ID	The ODB type of the array elements	5	1 int
4	Class Info OID	The class info OID if sub type is a NonNative Object. If not, this field does not exist	9	1 int
5	Name	Attribute name	9 / 13	variable
6	Is used in index	If attribute is used in an index	-	1 byte

if attribute is a Non Native Object : array

No	Field	Description	Position	Size
1	Is Native	Itis not native : 0	0	1 byte
2	Class Info OID	The class info OID of the attribute	1	1 Long
3	Name	Attribute name	9	variable
4	Is used in index	If attribute is used in an index	-	1 byte

Writing implemented in org.neodatis.odb.core.io.ObjectWriter.writeClassInfoBody(…)
Reading implemented in org.neodatis.odb.core.io.ObjectReader.readClassInfoBody(…)

Object Block

Object block

No	Field	Description	Position	Size
1	Block size	the size of the block	0	1 int
2	Block type	4 (org.neodatis.odb.core.io.BlockTypes.BLOCK_TYPE_NON_NATIVE_OBJECT)	1	1 byte
3	OID	The Object ID	5	1 Long
4	ClassInfo OID	The OID of the class of the object	13	1 Long
5	Previous Object OID	OID of the previous object	21	1 Long
6	Next Object OID	OID if the next Object	29	1 Long
7	Creation Date	The Object creation date	37	1 Long
8	Update Date	The Object update date	45	1 Long
9	Version number	Object version number	53	1 Int
10	Object reference	Not used yet. If this objects Is being referenced by others,then this field will be greater>0(v2.0), in version 2.x, it will point to an internal object of type ObjectReference that will have details on the references. This is to enable object integrity.: to enable object integrity	57	1 Long
11	External synchronization indicator	Not used yet. True if this object have been synchronized with main database, else false	65	1 boolean
12	Number of attributes	Number of attributes of the object	66	1 Int

Then write the attribute data.

First write a block of NbAttributes elements. Each element contains the id of the attribute (int) and the OID or position of the value of the attribute.

No	Field	Description	Position	Size
1	Attribute ID	this id is the ID of the attribute that was defined in the ClassInfo	0	1 int
2	Attribute value OID	How/where to find the value of the attribute. For native objects it is the physical position, for non native objects it is the OID (In this case, the OID, long value, is stored as a negative number to differ from position)	4	1 Long

Then actually writes the values of the attributes. Values of Native Objects are written in place. Non Native Objects are written at the end of the file.

Lists, arrays and Maps are written in the object block as native objects :

• Lists

No	Field	Description	Position	Size
1	Block type	The block type	-	1 int
2	Block size	The block size	-	1 byte
3	Odb Type ID	The odb type id (List)	-	1 int
4	Is Null?	To indicate if object is null	-	1 Boolean
5	Class name	The real Collection Class Name	-	String
6	Size	Size of the list	-	Long
7	Block of element identification	A block of OID of the elements of the list. If an element is native (ex: a string), the its position is stored instead of OID	-	block of long

• Native Attributes

No	Field	Description	Position	Size
1	Block Size	The block size, fix = 9	-	1 int
2	Block Type	The block type	-	1 byte
3	Attribute type	The odb type id	-	1 int
4	Is Null?	To indicate if object is null	-	1 Boolean
5	The attribute value	The bytes of the attribute value	-	-

BLOCK_TYPE_CLASS_HEADER = 1;
BLOCK_TYPE_CLASS_BODY = 2;
BLOCK_TYPE_NATIVE_OBJECT = 3;
BLOCK_TYPE_NON_NATIVE_OBJECT = 4;
BLOCK_TYPE_POINTER = 5;
BLOCK_TYPE_DELETED = 6;
BLOCK_TYPE_NON_NATIVE_NULL_OBJECT = 7;
BLOCK_TYPE_NATIVE_NULL_OBJECT = 77;
BLOCK_TYPE_COLLECTION_OBJECT = 8;
BLOCK_TYPE_ARRAY_OBJECT = 9;
BLOCK_TYPE_MAP_OBJECT = 10;
BLOCK_TYPE_IDS = 20;
BLOCK_TYPE_INDEX = 21;

ObjectTypes are defined in class ODBType.

Class/Method : org.neodatis.odb.impl.core.layers.layer3.engine.AbstractObjectWriter.writeCollection

Obs: Here we should first write all native objects and then write Non Native.

Writing implemented in org.neodatis.odb.core.io.ObjectWriter.writeNonNativeObjectInfo (…)
Reading implemented in org.neodatis.odb.core.io.ObjectReader.readNonNativeObjectInfoFromPosition(…)

String

Strings have a specific way to be written to file. A String in ODB have 2 size attributes:
1) the max size that have been reserved for the string (to enable storing a bigger string in the same place)
2) the real string size

The bytes that represent the string are encoded using the encoding defined by the Configuration.getDatabaseCharacterEncoding(). Default is ISO8859-1.

No	Field	Description	Position	Size
1	Total size	Total String size	-	1 int
2	Real size	Real String size	-	1 int
3	The String	The bytes of the string	-	variable

The encoding & decoding are implemented in public byte [] stringToByteArray(String s,boolean withSize, int totalSpace, boolean withEncoding) & public String byteArrayToString(byte [] bytes,boolean hasSize, boolean useEncoding) in class org.neodatis.odb.impl.core.layers.layer3.engine.DefaultByteArrayConverter.

Implementation

This format is implemented by the the org.neodatis.odb.core.io.Writer for writing and org.neodatis.odb.core.io.ObjectReader for reading.

org.neodatis.odb.core.io.StorageEngineConstant contains some constant value for each field position.