Ingres CL JF
From Ingres Community Wiki
|
Ingres Compatability Library |
Compatibility Library Specification - JF
Abstract
This is the specification of the JF facility provided by the compatibility library for access to journal files.
Revision: 1.1, 11-Nov-1997
Document History
- Revision 1.1, last modified 11-Nov-1997
- Converted to HTML.
- Revision 1.0, last modified 29-Aug-91.
- Fixed typos.
Specification
Introduction
JF provides file operations required to support journaling and online-checkpoint.
The current JF routines are used to open/close/read/write/merge/update the journal and to open/close/read/write/update the dump files, which are needed to back out the database changes to the point where online backup starts.
Library
CL
Intended Uses
The JF services are intended for use by the DMF code to manage the set of journal files for a database. A journal file contains a history of all journaled changes made to a database. A destroyed database can be recovered from the contents of the journal file. In conjunction with a checkpoint, a subset of the journal files can be used to recover a database. These files are named by DMF and reside at an INGRES location specified for journals by ACCESSDB.
Assumptions
The journal file can be written to disk or tape. The current implementation only considers disks, but future expansion to include the use of tapes has been considered.
Only one open instance per journal file is allowed.
Definitions
| Block sequence | Each block written to the journal file is assumed to be assigned a sequence number. This sequence number is used to determine whether this block has been seen before. |
| Journal file | A journal file is a operating system file that contains a sequence of database actions performed by DMF against a database over a certain period of time. A journal file name must be 12 bytes long. The format is always Jnnnmmmm.JNL, where nnn and mmmm are numbers. These numbers keep track of the versions of the journal files in case there are multiple versions since a checkpoint. These files reside at a location specified by the user when a database is created or by ACCESSDB. In other words the path will be the value returned from LOingpath when the what parameter is set to LO_JNL. The file is extended automatically by writing a block past the EOF, it may not contain "holes", and does not need to support concurrent opens. |
| Dump file | A dump file is an operating system file that contains a sequence of database before images performed by DMF against a database during online backup. A dump file name must be at least 12 bytes long. The format is always Dnnnmmmm.DMP, where nnn and mmmm are numbers. These numbers keep track of the versions of the dump files. In case there are multiple versions since a checkpoint. These files reside at a location specified by the user when a database is created or by ACCESSDB. In other words, the path will be the value returned from LOingpath when the what parameter is set to LO_DMP. The file is extended automatically by writing a block past the EOF; it may not contain "holes," and does not need to support concurrent opens. |
| Journal block | A journal file is created with a certain block size. This block size governs the size of each transfer to and from the journal file. The contents of the block are completely ignored by this version of JF. It only reads and writes the contents, but does not analyze it. Journal i/o is performed by requesting logical block numbers. The first logical block of a file is 1. Logical block zero is never used. Physical blocks of a journal file do not have to map directly onto logical blocks. That is if a particular implementation of JF needed to reserve 10 physical blocks at the beginning of the file for control information, then physical block 11 would map to logical block 1 which would be the first logical block in the file. |
| Dump block | A dump file is created with a certain block size. This block size governs the size of each transfer to and from the dump file. The contents of the block are completely ignored by this version of JF. It only reads and writes the contents, but does not analyze it. Dump i/o is performed by requesting logical block numbers. The first logical block of a file is 1. Logical block zero is never used. Physical blocks of a dump file do not have to map directly onto logical blocks. That is, if a particular implementation of JF needed to reserve 10 physical blocks at the beginning of the file for control information, then physical block 11 would map to logical block 1 which would be the first logical block in the file. |
| Journal record | The journal file actually contains a sequence of journal records. The mapping of records to blocks is considered a function of the client. In future versions of JF, the record mapping may be included. |
| Dump record | The dump file actually contains a sequence of dump records. The mapping of records to blocks is considered a function of the client. In future versions of JF, the record mapping may be included. |
| Journal set | A single journal file contains a sequence of database changes for a specific time period. All journal files from the first one created for a database to the last one used constitute a journal set. This is the complete history of changes. |
| Dump set | A single dump file contains a sequence of database changes for a specific time period. All dump files from the first one created for a database to the last one used constitute a dump set. This is the complete history of changes. |
Concepts
The JF services are very simple and straight forward in the currrent form. When the ability to write to tape and the inclusion of record blocking are added the complexity of these routines will increase substantially. Most of the increase in complexity comes from the manipulation of tape volumes.
A journal or dump file is like any normal file, other than the fact that each block is tagged with a header that records information used to enhance the reliability of the software.
Journal or dump blocks are not necessarily full. They can be partially filled. JFtruncate is used to claim wasted space from dormant journal files only.
Journal or dump blocks are never logically rewritten on disks and never physical rewritten on tapes. Rewriting a tape block is not supported on a lot of tape equipment, as well as being less reliable when used (Unless an intelligent tape controller is using a mechanism similar to our block sequence writing.)
The block size of a journal or dump file can range from 4096 to 32768 in powers of 2.
The maximum number of blocks in a journal or dump file is 2^32-1.
The maximum number of journal or dump files in 9,999,999.
A JFIO control block is initialized when a journal or dump file is opened through a call to JFopen. The same control block must be used on all subsequent calls to JF functions until the journal or dump file is closed through a call to JFclose. The JF functions may update data in the JFIO during any call.
Header File <jf.h>
The header file <jf.h> must be included before using any of the functions provided. It defines the following.
JFIO - Journal or dump file context
This structure contains information initialized and used by the JF routines to manage an open journal or dump file. The actual structure is allocated by the caller. This is the structure used on VAX/VMS. This structure is a sample. Client code may not refer to any field in this structure.
typedef struct _JFIO
{
i4 jf_ascii_id; /* Visible identifier. */
#define JF_ASCII_ID 'JFIO'
i4 jf_allocate; /* Number of
** allocated VMS block. */
i4 jf_blk_size; /* Block size for transfer. */
i4 jf_log2_bksz; /* Log 2 of block size. */
i4 jf_blk_cnt; /* VMS blocks in a JF block. */
i4 jf_cur_blk; /* Current JF block number. */
i4 jf_channel; /* VMS I/O channel. */
i4 jf_extra;
short jf_fid[4]; /* File identifier. */
} JFIO;
Executable Interface
The following functions are provided.
JFclose - close journal or dump file
This routine closes the journal or dump file described by the JFIO context passed to this routine. This routine must flush any outstanding asynchronous writes.
Inputs:
| jfio | Pointer to journal or dump file context. |
Outputs:
| jfio | Pointer to journal file context. |
| err_code | Operating system error. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFclose(jfio, err_code) JFIO *jfio; CL_ERR_DESC *err_code;
JFcreate - create a journal or dump file
This routine creates and allocates a journal or dump file for processing. This file is created in the area specified by device/path name. The file cannot be left open.
Inputs:
| jf_io | Journal or dump file context. This is used as a work area. |
| device | Pointer to device/path name. |
| l_device | Length of device/path name. |
| file | Pointer to file name. |
| l_file | Length of file name. |
| bksz | Block size. Must be one of:
4096, 8192, 16384, 32768 for journal file; 2048 for dump file. |
| blkcnt | Number of blocks to pre-allocate. This may be extended. |
Outputs:
| err_code | Reason for error return status. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFcreate(jf_io, device, l_device, file, l_file, bksz, blkcnt, err_code) JFIO *jf_io; char *device; u_i4 l_device; char *file; u_i4 l_file; u_i4 bksz; i4 blkcnt; CL_ERR_DESC *err_code;
JFdelete - delete a journal or dump file
This routine deletes a journal or dump file. The file must be closed before this routine is called.
Inputs:
| jf_io | Journal or dump file context. This is used as a work area. |
| path | Pointer to device path. |
| l_path | Length of path. |
| filename | Pointer to filename. |
| l_filename | Length of filename. |
Outputs:
| sys_err | Operating system error. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFdelete(jf_io, path, l_path, filename, l_filename, sys_err) JFIO *jf_io; char *path; u_i4 l_path; char *filename; u_i4 l_filename; CL_ERR_DESC *sys_err;
JFdircreate - create a journal or dump directory
The JFdircreate will take a path name and directory name and create a directory in this location.
Inputs:
| f | Pointer to file context area. This is used as a work area. |
| path | Pointer to the path name. |
| pathlength | Length of path name. |
| dirname | Pointer to the directory name. |
| dirlength | Length of directory name. |
Outputs:
| err_code | Pointer to a variable used to return operating system errors. |
Returns:
| JF_EXISTS | Already exists. |
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFdircreate(f,path, pathlength, dirname, dirlength, err_code) JFIO *f; char *path; u_i4 pathlength; char *dirname; u_i4 dirlength; CL_ERR_DESC *err_code;
JFdirdelete - delete a journal or dump directory
The JFdirdelete routine is used to delete a Journal or Dump File directory. The name of the directory to delete should be specified as path and dirname, where dirname does not include a type qualifier or version and the path does not include the directory name (which it normally does for other JF calls). This call should fail with JF_BAD_DELETE if any files still exists in the directory.
Inputs:
| f | Pointer to file context. |
| path | Pointer to the area containing the path name for this directory. |
| pathlength | Length of path name. |
| dirname | Pointer to the area containing the directory name. |
| dirlength | Length of directory. |
Outputs:
| err_code | Pointer to a variable used to return system specific errors. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFdirdelete(f, path, pathlength, dirname, dirlength, err_code) JFIO *f; char *path; u_i4 pathlength; char *dirname; u_i4 dirlength; CL_ERR_DESC *err_code;
JFlistfile - list all files in a journal or dump directory
The JFlistfile routine will list all files that exist in the directory(path) specified. This routine expects as input a function to call for each file found. This insures that all resources tied up by this routine will eventually be freed. The files are not listed in any specific order. The function passed to this routine must return OK if it wants to continue with more files, or a value not equal to OK if it wants to stop listing. If the function returns anything other than OK, then JFlistfile will return JF_END_FILE. The function must have the following definition:
STATUS func(arg_list, filename, filelength, err_code) PTR arg_list; Pointer to argument list char *filename; Pointer to directory name. u_i4 filelength; Length of directory name. CL_ERR_DESC *err_code; Pointer to operating system error codes (if applicable).
Inputs:
| f | Pointer to file context area. |
| path | Pointer to the path name. |
| pathlength | Length of path name. |
| func | Function to pass the directory entries to. |
| arg_list | Pointer to function argument list. |
Outputs:
| err_code | Pointer to a variable used to return system specific errors. |
Returns:
| Never returns OK. | |
| JF_END_FILE | All files listed, otherwise system specific error status. |
Definition:
STATUS JFlistfile(f,path, pathlength, func, arg_list, err_code) JFIO *f; char *path; u_i4 pathlength; STATUS (*func)(); PTR arg_list; CL_ERR_DESC *err_code;
JFopen - open a journal or dump file
This routine will open the journal or dump file described by the device and file name. The block size specified must match the block size used to create the file. (This value should never change.) If an implementation can detect different blocksizes, it should return an error. The number of the last block written is returned. Blocks are numbered starting at 1. A return value of zero denotes an empty file. Must be able to continue writing to the next block beyond the current end of file. If the file cannot be found, then JF must return JF_NOT_FOUND
Inputs:
| jfio | Journal or dump file I/O context. |
| device | Pointer to device/path name. |
| l_device | Length of device/path name. |
| file | Pointer to file name. |
| l_file | Length of file name. |
| bksz | Block size. Must match the block size specified in JFcreate. |
Outputs:
| jfio | Journal or dump file I/O context. |
| eof_block | Last block number updated by JFupdate. |
| err_code | Operating system error. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
| JF_NOT_FOUND | Could not find file. |
Definition:
STATUS JFopen(jfio, device, l_device, file, l_file, bksz, eof_block, err_code) JFIO *jfio; char *device; u_i4 l_device; char *file; u_i4 l_file; u_i4 bksz; i4 *eof_block; CL_ERR_DESC *err_code;
JFread - read block from journal or dump file
Read block from the journal or dump file.
Inputs:
| jfio | Pointer to journal or dump context set up by JFopen. |
| buffer | Pointer to journal or dump block buffer. |
| block | Block number to read. Ignored if this is a tape device. |
Outputs:
| jfio | Updated journal or dump context. |
| err_code | system specific error on failure. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFread(jfio, buffer, block, err_code) JFIO *jfio; PTR buffer; i4 block; CL_ERR_DESC *err_code;
JFtruncate - truncates journal or dump files at checkpoint time
This routine is intended to return to the system the disk space that was allocated, but left unused by the previous checkpoint's journal or dump files. This routine uses the current EOF block marker, as determined by JFupdate, and truncates the journal or dump file to this position. The file must be open. This may be a NO-OP.
Inputs:
| jfio | Pointer to journal or dump file context. |
Outputs:
| err_code | system specific error. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFtruncate(jfio, err_code) JFIO *jfio; CL_ERR_DESC *err_code;
JFupdate - update journal or dump file position to disk
Update the position in the journal or dump file. This keeps track of the current block at end of file. This value is returned to JFopen. This is a seek position that is valid across system crashes.
Inputs:
| jfio | Pointer to journal or dump file context. |
Outputs:
| err_code | Operating system error. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFupdate(jfio, err_code) JFIO *jfio; CL_ERR_DESC *err_code;
JFwrite - write block to journal or dump file
This routine allocates space for the next block (if needed) and writes the block. The writes can be buffered for delayed writes, i.e. they can be asynchronous.
Inputs:
| jfio | Pointer to journal or dump context. |
| buffer | Pointer to journal or dump block to write. |
| block | Block number to write. Ignored if this is a tape device. |
Outputs:
| err_code | system specific error. |
Returns:
| OK | if operation succeeded, otherwise system specific error status. |
Definition:
STATUS JFwrite(jfio, buffer, block, err_code) JFIO *jfio; PTR buffer; i4 block; CL_ERR_DESC *err_code;
Examples
JFIO jfio;
DB_STATUS status;
i4 i;
i4 number;
char file[12];
char device[64];
i4 l_device;
CL_ERR_DESC sys_err;
/* Construct the file name. */
number = 1;
file[0] = 'J';
file[1] = ((number / 10000000) % 10) + '0';
file[2] = ((number / 100000)% 10) + '0';
file[3] = ((number / 10000) % 10) + '0';
file[4] = ((number / 1000) % 10) + '0';
file[5] = ((number / 100) % 10) + '0';
file[6] = ((number / 10) % 10) + '0';
file[7] = (number % 10) + '0';
file[8] = '.';
file[9] = 'J';
file[10] = 'N';
file[11] = 'L';
/* Create a journal file using 1024 8K blocks. */
status = JFcreate(jfio, device, l_device, file,
sizeof(file), 8192, 1024, &sys_err);
if (status)
error();
jf_sequence = 1;
status = JFopen(&jfio, device, l_device, &file, sizeof(file),
8192, &jf_sequence, &sys_err);
if (status != OK)
error();
/* Write a bunch of blocks to the journal file. */
for (i = 1; i <= 10; i++)
{
status = JFwrite(&jfio, &page, i, &err_code);
if (status != OK)
error();
}
/* Update block sequence for disk based file. */
status = JFupdate(&jfio, 10, &sys_err);
if (status != OK)
error();
/* Now read the pages back again. */
for (i = 1; i <= 10; i++)
{
status = JFread(&jfio, &page, i, &sys_err);
if (status != OK)
error();
}
/* Close the journal file. */
status = JFclose(&jfio, &sys_err);
if (status != OK)
error();
|
Ingres Compatability Library |
