Tokyo Cabinet

Tokyo Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Records are organized in hash table, B+ tree, or fixed-length array.

The following access methods are provided to the database:

Moreover, traversal access to every key are provided, although the order is arbitrary.

As for database of fixed-length array, records are stored with unique natural numbers. It is impossible to store two or more records with a key overlaps. Moreover, the length of each record is limited by the specified length. Provided operations are the same as ones of hash database.

Tokyo Cabinet is written in the C language, and provided as API of C, Perl, Ruby, and Lua.

Tokyo Cabinet is available on platforms which have API conforming to C99 and POSIX. Tokyo Cabinet is licensed under the GNU Lesser General Public License.

* * *

Tokyo Cabinet provides two modes to connect to a database: "reader" and "writer". A reader can perform retrieving but neither storing nor deleting. A writer can perform all access methods.

Exclusion control between processes is performed when connecting to a database by file locking.

While a writer is connected to a database, neither readers nor writers can be connected.

While a reader is connected to a database, other readers can be connect, but writers can not.

According to this mechanism, data consistency is guaranteed with simultaneous connections in multitasking environment.

Tokyo Cabinet provides API of the C language and it is available by programs conforming to the C89 (ANSI C) standard or the C99 standard.

As the header files of Tokyo Cabinet are provided as 'tcutil.h', 'tchdb.h', and `tcbdb.h', applications should include one or more of them accordingly to use the API.

As the library is provided as 'libtokyocabinet.a' and `libtokyocabinet.so' and they depends 'libz.so', 'librt.so', 'libpthread.so', 'libm.so', and 'libc.so'

linker options `-ltokyocabinet', `-lz', '-lbz2', `-lrt', `-lpthread', '-lm', and '-lc' are required for build command. A typical build command is the following.

gcc -I/usr/local/include tc_example.c -o tc_example \\ -L/usr/local/lib -ltokyocabinet -lz -lbz2 -lrt -lpthread -lm -lc

You can also use Tokyo Cabinet in programs written in C++. Because each header is wrapped in C linkage ('extern "C"' block), you can simply include them into your C++ programs.



----------------------------------------------------------------------

    $> ttserver book.tct



tctmgr - the command line utility of the table database API

tc  - "tokyo cabinet"
t   - linear table 
mgr - manager


 $> tctmgr inform -nl boss_book.tcl

 $> tctmgr list -nl boss_book.tcl 

 $> tctmgr get -nl boss_book.tcl 

 $> tctmgr put -nl boss_book.tcl  [cols ...]

 $> tctmgr out -nl boss_book.tcl    # dangerous !!! delete without recovery !!!



	tokyo stuff

content	
 .tch   hash
 .tct   table-like
 .tcb   b-tree


------------------------- Cabinet

 Tokyo Cabinet is a library of routines for managing a database
 
 database is a simple data file containing records, each is a pair of a key and a value
 every key and value is serial bytes with variable length
 both binary data and character string can be used as a key and a value
 there is neither concept of data tables nor data types
 records are organized in hash table, B+ tree, or fixed-length array


Tokyo Tyrant

Tokyo Tyrant is a package of network interface to Tokyo Cabinet db

though the DBM has high performance, you might bother in case that

  • multiple processes share the same database, or
  • remote processes access the database

    thus, Tokyo Tyrant is provided for concurrent and remote connections to Tokyo Cabinet

    
    Like BerkeleyDB, TC gives you low-level database operations and allows you to build
    your own very fast data access operations
    
    unlike BerkeleyDB, TC has several functional layers on top of TC which provide
     - network access,
     - schema-less document-oriented storage (a la CouchDB) and
     - supported bindings for many languages
     
    
    ----------------- 
    CLI
    
    
     $> tctmgr list -nl /path/to/database/file.tct
    
     $> tctmgr out -nl /path/to/database/file.tct some_key_name  # DANGEROS
    
    
    
    
    
    
    
    --------------tcrmgr - the command line utility of the remote database API
    
    you can delete all records of your database via tokyotyrant with the vanish command, e.g.
     $> tcrmgr vanish [-port num] host
    
    for test purposes:
     $> time for x in {1..2000}; do tcrmgr get localhost somekey ; done
    
    $> ttserver -host 127.0.0.1 -port 9000 /path/to/your/database/yourdbname.tct
    $> tcrmgr get -port 9000 localhost yourkeyid 
    
    
    
    ----------------
    API
    
    
    tctmgr create [-tl] [-td|-tb|-tt|-tx] path [bnum [apow [fpow]]]
     Create a database file.
    
    tctmgr inform [-nl|-nb] path
     Print miscellaneous information to the standard output.
    
    tctmgr put [-nl|-nb] [-sx] [-dk|-dc|-dai|-dad] path pkey [cols ...]
     Store a record.
    
    tctmgr out [-nl|-nb] [-sx] path pkey
     Remove a record.
    
    tctmgr get [-nl|-nb] [-sx] [-px] [-pz] path pkey
     Print the value of a record.
    
    tctmgr list [-nl|-nb] [-m num] [-pv] [-px] [-fm str] path
     Print the primary keys of all records, separated by line feeds.
    
    tctmgr search [-nl|-nb] [-ord name type] [-m num] [-sk num] [-kw] [-pv] [-px] [-ph]
     [-bt num] [-rm] [-ms type] path [name op expr ...]
     Print records matching conditions, separated by line feeds.
    
    tctmgr optimize [-tl] [-td|-tb|-tt|-tx] [-tz] [-nl|-nb] [-df] path [bnum [apow [fpow]]]
     Optimize a database file.
    
    tctmgr setindex [-nl|-nb] [-it type] path name
     Set the index of a column.
    
    tctmgr importtsv [-nl|-nb] [-sc] path [file]
     Store records of TSV in each line of a file.
    
    tctmgr version
     Print the version information of Tokyo Cabinet.
    
    
    Options feature the following:
     -mt     : call the function `tctdbsetmutex'.
     -tl     : enable the option `TDBTLARGE'.
     -td     : enable the option `TDBTDEFLATE'.
     -tb     : enable the option `TDBTBZIP'.
     -tt     : enable the option `TDBTTCBS'.
     -tx     : enable the option `TDBTEXCODEC'.
     -rc num : specify the number of cached records.
     -lc num : specify the number of cached leaf pages.
     -nc num : specify the number of cached non-leaf pages.
     -xm num : specify the size of the extra mapped memory.
     -df num : specify the unit step number of auto defragmentation.
     -ip     : create the number index for the primary key.
     -is     : create the string index for the column "str".
     -in     : create the number index for the column "num".
     -it     : create the string index for the column "type".
     -if     : create the token inverted index for the column "flag".
     -ix     : create the q-gram inverted index for the column "text".
     -nl     : enable the option `TDBNOLCK'.
     -nb     : enable the option `TDBLCKNB'.
     -pn num : specify the number of patterns.
     -dai    : use the function `tctdbaddint' instead of `tctdbputcat'.
     -dad    : use the function `tctdbadddouble' instead of `tctdbputcat'.
     -rl     : set the length of values at random.
     -ru     : select update operations at random.
    
    
    =============
    
     tcbmgr - the command line utility of the B+ tree database API
            
    
            `path'   the path of a database file.
            `lmemb'  the number of members in each leaf page.
            `nmemb'  the number of members in each non-leaf page.
            `bnum'   the number of buckets.
            `apow'   the power of the alignment.
            `fpow'   the power of the free block pool.
            `key'    the key of a record.
            `value'  the value of a record.
            `file'   the input file.
    
    tcbmgr create [-cd|-ci|-cj] [-tl] [-td|-tb|-tt|-tx] path [lmemb [nmemb [bnum [apow [fpow]]]]]
    Create a database file.
    
    tcbmgr inform [-nl|-nb] path
    Print miscellaneous information to the standard output
    
    tcbmgr put [-cd|-ci|-cj] [-nl|-nb] [-sx] [-dk|-dc|-dd|-db|-dai|-dad] path key value
    Store a record
    
    tcbmgr out [-cd|-ci|-cj] [-nl|-nb] [-sx] path key
    Remove a record
    
    tcbmgr get [-cd|-ci|-cj] [-nl|-nb] [-sx] [-px] [-pz] path key
    Print the value of a record
    
    tcbmgr list [-cd|-ci|-cj] [-nl|-nb] [-m num] [-bk] [-pv] [-px] [-j str] [-rb bkey ekey] [-fm str] path
    Print keys of all records, separated by line feeds
    
    tcbmgr optimize [-cd|-ci|-cj] [-tl] [-td|-tb|-tt|-tx] [-tz] [-nl|-nb] [-df] path
       [lmemb [nmemb [bnum [apow [fpow]]]]]
    Optimize a database file
    
    tcbmgr importtsv [-nl|-nb] [-sc] path [file]
    Store records of TSV in each line of a file
    
    tcbmgr version
    Print the version information of Tokyo Cabinet
    
    
    Options feature the following.
            -cd           : use the comparison function `tccmpdecimal'.
            -ci           : use the comparison function `tccmpint32'.
            -cj           : use the comparison function `tccmpint64'.
            -tl           : enable the option `BDBTLARGE'.
            -td           : enable the option `BDBTDEFLATE'.
            -tb           : enable the option `BDBTBZIP'.
            -tt           : enable the option `BDBTTCBS'.
            -tx           : enable the option `BDBTEXCODEC'.
            -nl           : enable the option `BDBNOLCK'.
            -nb           : enable the option `BDBLCKNB'.
            -sx           : the input data is evaluated as a hexadecimal data string.
            -dk           : use the function `tcbdbputkeep' instead of `tcbdbput'.
            -dc           : use the function `tcbdbputcat' instead of `tcbdbput'.
            -dd           : use the function `tcbdbputdup' instead of `tcbdbput'.
            -db           : use the function `tcbdbputdupback' instead of `tcbdbput'.
            -dai          : use the function `tcbdbaddint' instead of `tcbdbput'.
            -dad          : use the function `tcbdbadddouble' instead of `tcbdbput'.
            -px           : the output data is converted into a hexadecimal data string.
            -pz           : do not append line feed at the end of the output.
            -m num        : specify the maximum number of the output.
            -bk           : perform backword scanning.
            -pv           : print values of records also.
            -j str        : specify the key where the cursor jump to.
            -rb bkey ekey : specify the range of keys.
            -fm str       : specify the prefix of keys.
            -tz           : enable the option `UINT8_MAX'.
            -df           : perform defragmentation only.
            -sc           : normalize keys as lower cases.
    
    
    This command returns 0 on success, another on failure.