Package com.mckoi.database

Class V2MasterTableDataSource

  • public final class V2MasterTableDataSource
extends MasterTableDataSource
    A MasterTableDataSource that is backed by a non-shared com.mckoi.store.Store object. The store interface allows us a great deal of flexibility because we can map a store around different underlying devices. For example, a store could map to a memory region, a memory mapped file, or a standard random access file.

    Unlike V1MasterTableDataSource, this manages data and index information in a single store which can be backed by a single file in the file system.

    The structure of the store comprises of a header block that contains the following information; 

           HEADER BLOCK
   +-------------------------------+
   | version                       |
   | table id                      |
   | table sequence id             |
   | pointer to DataTableDef       |
   | pointer to DataIndexSetDef    |
   | pointer to index block        |
   | LIST BLOCK HEADER pointer     |
   +-------------------------------+

    Each record is comprised of a header which contains offsets to the fields in the record, and a serializable of the fields themselves.

    • Field Detail

      • file_name

        private java.lang.String file_name
        The file name of this store in the conglomerate path.

      • store

        private Store store
        The backing store object.

      • index_store

        private IndexSetStore index_store
        An IndexSetStore object that manages the indexes for this table.

      • sequence_id

        private long sequence_id
        The current sequence id.

      • index_header_p

        private long index_header_p
        Points to the index header area.

      • list_header_p

        private long list_header_p
        Points to the block list header area.

      • header_area

        private MutableArea header_area
        The header area itself.

      • list_structure

        private FixedRecordList list_structure
        The structure that manages the pointers to the records.

      • first_delete_chain_record

        private long first_delete_chain_record
        The first delete chain element.

      • has_shutdown

        private boolean has_shutdown
        Set to true when the VM has shutdown and writes should no longer be possible on the object.

      • s_run_file_hits

        private short s_run_file_hits

    • Method Detail

      • getDOut

        private static java.io.DataOutputStream getDOut​(java.io.OutputStream out)
        Convenience - wraps the given output stream around a buffered data output stream.

      • getDIn

        private static java.io.DataInputStream getDIn​(java.io.InputStream in)
        Convenience - wraps the given input stream around a buffered data input stream.

      • setupInitialStore

        private void setupInitialStore()
                        throws java.io.IOException
        Sets up an initial store (should only be called from the 'create' method).
        Throws:
        java.io.IOException

      • readStoreHeaders

        private void readStoreHeaders()
                       throws java.io.IOException
        Read the store headers and initialize any internal object state. This is called by the 'open' method.
        Throws:
        java.io.IOException

      • create

        void create​(int table_id,
            DataTableDef table_def)
     throws java.io.IOException
        Create this master table in the file system at the given path. This will initialise the various file objects and result in a new empty master table to store data in.
        Throws:
        java.io.IOException

      • exists

        boolean exists​(java.lang.String identity)
        throws java.io.IOException
        Returns true if the master table data source with the given source identity exists.
        Throws:
        java.io.IOException

      • open

        public void open​(java.lang.String file_name)
          throws java.io.IOException
        Opens an existing master table from the file system at the path of the conglomerate this belongs to. This will set up the internal state of this object with the data read in.
        Throws:
        java.io.IOException

      • close

        void close​(boolean pending_drop)
    throws java.io.IOException
        Closes this master table in the file system. This frees up all the resources associated with this master table.

        This method is typically called when the database is shut down.

        Throws:
        java.io.IOException

      • copy

        void copy​(int table_id,
          MasterTableDataSource src_master_table,
          IndexSet index_set)
   throws java.io.IOException
        Creates a new v2 master table data source that is a copy of the given MasterTableDataSource object.
        Parameters:
        table_id - the table id to given the new table.
        src_master_table - the table to copy.
        index_set - the view of the table to be copied.
        Throws:
        java.io.IOException

      • writeRecordToStore

        private long writeRecordToStore​(RowData data)
                         throws java.io.IOException
        Writes a record to the store and returns a pointer to the area that represents the new record. This does not manipulate the fixed structure in any way. This method only allocates an area to store the record and serializes the record. It is the responsibility of the callee to add the record into the general file structure.

        Note that if the RowData contains any references to Blob objects then a reference count to the blob is generated at this point.

        Throws:
        java.io.IOException

      • copyRecordFrom

        private void copyRecordFrom​(MasterTableDataSource src_master_table,
                            int record_id)
                     throws java.io.IOException
        Copies the record at the given index in the source table to the same record index in this table. Note that this may need to expand the fixed list record heap as necessary to copy the record into the given position. The record is NOT copied into the first free record position.
        Throws:
        java.io.IOException

      • removeAllBlobReferencesForRecord

        private void removeAllBlobReferencesForRecord​(long record_p)
                                       throws java.io.IOException
        Removes all blob references in the record area pointed to by 'record_p'. This should only be used when the record is be reclaimed.
        Throws:
        java.io.IOException

      • dropAllBlobReferences

        private void dropAllBlobReferences()
                            throws java.io.IOException
        Scans the table and drops ALL blob references in this table. This is used when a table is dropped when is still contains elements referenced in the BlobStore. This will decrease the reference count in the BlobStore for all blobs. In effect, this is like calling 'delete' on all the data in the table.

        This method should only be called when the table is about to be deleted from the file system.

        Throws:
        java.io.IOException

      • scanForLeaks

        public void scanForLeaks()
                  throws java.io.IOException
        Looks for any leaks in the file. This works by walking through the file and index area graph and 'remembering' all areas that were read. The store is then checked that all other areas except these are deleted.

        Assumes the master table is open.

        Throws:
        java.io.IOException

      • checkAndRepair

        public void checkAndRepair​(java.lang.String file_name,
                           UserTerminal terminal)
                    throws java.io.IOException
        Performs a complete check and repair of the table. The table must not have been opened before this method is called. The given UserTerminal parameter is an implementation of a user interface that is used to ask any questions and output the results of the check.
        Throws:
        java.io.IOException

      • checkAndRepairRecord

        private boolean checkAndRepairRecord​(int row_index,
                                     java.util.List all_areas,
                                     UserTerminal terminal)
                              throws java.io.IOException
        Checks and repairs a record if it requires repairing. Returns true if the record is valid, or false otherwise (record is/was deleted).
        Throws:
        java.io.IOException

      • growListStructure

        private void growListStructure()
                        throws java.io.IOException
        Grows the list structure to accomodate more entries. The new entries are added to the free chain pool. Assumes we are synchronized over list_structure.
        Throws:
        java.io.IOException

      • addToRecordList

        private long addToRecordList​(long index,
                             long record_p)
                      throws java.io.IOException
        Adds a record to the given position in the fixed structure. If the place is already used by a record then an exception is thrown, otherwise the record is set.
        Throws:
        java.io.IOException

      • addToRecordList

        private long addToRecordList​(long record_p)
                      throws java.io.IOException
        Finds a free place to add a record and returns an index to the record here. This may expand the record space as necessary if there are no free record slots to use.
        Throws:
        java.io.IOException

      • getSourceIdent

        java.lang.String getSourceIdent()
        Description copied from class: MasterTableDataSource
        Returns a string that uniquely identifies this table within the conglomerate context. For example, the filename of the table. This string can be used to open and initialize the table also.
        Specified by:
        getSourceIdent in class MasterTableDataSource

      • writeRecordType

        int writeRecordType​(int row_index,
                    int row_state)
             throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Sets the record type for the given record in the table and returns the previous state of the record. This is used to change the state of a row in the table.
        Specified by:
        writeRecordType in class MasterTableDataSource
        Throws:
        java.io.IOException

      • readRecordType

        int readRecordType​(int row_index)
            throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Reads the record state for the given record in the table.
        Specified by:
        readRecordType in class MasterTableDataSource
        Throws:
        java.io.IOException

      • recordDeleted

        boolean recordDeleted​(int row_index)
               throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Returns true if the record with the given index is deleted from the table. A deleted row can not be read.
        Specified by:
        recordDeleted in class MasterTableDataSource
        Throws:
        java.io.IOException

      • rawRowCount

        int rawRowCount()
         throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Returns the raw count or rows in the table, including uncommited, committed and deleted rows. This is basically the maximum number of rows we can iterate through.
        Specified by:
        rawRowCount in class MasterTableDataSource
        Throws:
        java.io.IOException

      • internalDeleteRow

        void internalDeleteRow​(int row_index)
                throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Removes the row at the given index so that any resources associated with the row may be immediately available to be recycled.
        Specified by:
        internalDeleteRow in class MasterTableDataSource
        Throws:
        java.io.IOException

      • createIndexSet

        IndexSet createIndexSet()
        Description copied from class: MasterTableDataSource
        Creates and returns an IndexSet object that is used to create indices for this table source. The IndexSet represents a snapshot of the table and the given point in time.

        NOTE: Not synchronized because we synchronize in the IndexStore object.

        Specified by:
        createIndexSet in class MasterTableDataSource

      • internalAddRow

        int internalAddRow​(RowData data)
            throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Adds a new row to this table and returns an index that is used to reference this row by the 'getCellContents' method.

        Note that this method will not effect the master index or column schemes. This is a low level mechanism for adding unreferenced data into a conglomerate. The data is referenced by committing the change where it eventually migrates into the master index and schemes.

        Specified by:
        internalAddRow in class MasterTableDataSource
        Throws:
        java.io.IOException

      • skipStream

        private void skipStream​(java.io.InputStream in,
                        long amount)
                 throws java.io.IOException
        Throws:
        java.io.IOException

      • internalGetCellContents

        TObject internalGetCellContents​(int column,
                                int row)
        Description copied from class: MasterTableDataSource
        Returns the cell contents of the given cell in the table. It is the responsibility of the implemented method to perform caching as it deems fit. Some representations may not require such extensive caching as others.
        Specified by:
        internalGetCellContents in class MasterTableDataSource

      • setUniqueID

        void setUniqueID​(long value)
        Description copied from class: MasterTableDataSource
        Sets the unique id for this store. This must only be used under extraordinary circumstances, such as restoring from a backup, or converting from one file to another.
        Specified by:
        setUniqueID in class MasterTableDataSource

      • dispose

        void dispose​(boolean pending_drop)
      throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Disposes of all in-memory resources associated with this table and invalidates this object. If 'pending_drop' is true then the table is to be disposed pending a call to 'drop'. If 'pending_drop' is true then any persistant resources that are allocated may be freed.
        Specified by:
        dispose in class MasterTableDataSource
        Throws:
        java.io.IOException

      • drop

        boolean drop()
      throws java.io.IOException
        Description copied from class: MasterTableDataSource
        Disposes and drops this table. If the dispose failed for any reason, it returns false, otherwise true. If the drop failed, it should be retried at a later time.
        Specified by:
        drop in class MasterTableDataSource
        Throws:
        java.io.IOException

      • shutdownHookCleanup

        void shutdownHookCleanup()
        Description copied from class: MasterTableDataSource
        Called by the 'shutdown hook' on the conglomerate. This method should block until the table can by put into a safe mode and then prevent any further access to the object after it returns. It must operate very quickly.
        Specified by:
        shutdownHookCleanup in class MasterTableDataSource

      • isWorthCompacting

        boolean isWorthCompacting()
        Description copied from class: MasterTableDataSource
        Returns true if a compact table is necessary. By default, we return true however it is recommended this method is overwritten and the table tested.
        Overrides:
        isWorthCompacting in class MasterTableDataSource

      • toString

        public java.lang.String toString()
        For diagnostic.
        Overrides:
        toString in class java.lang.Object