Introduction and Examples

CoreAPI abstracts various Database Technologies (FAME, Oracle, Cassandra, Scylla, MsSQL, MySql, PostgreSQL, Sysbase) with a modular backend that allows new technologies to be integrated into CoreAPI without changing client code. It also allows for extended functions that a particular backend may support such as FAME 4/GL code, automatic formula resolution and such extensions that are presented by the backends.

Configuration

The configuration of CoreAPI is driven by the TOM_HOME environment variable and should be set to the path where the tom-environment.xml configuration file lives. By default this will be under $HOME/.tom.

An example tom-environment.xml configuration file is included below. The settings should be clear, below for 3 example repositories. Repositories can include the physical location of databases or the network connection info to hook into a remote or local server process.

The sample tom-environment.xml configuration will need little modification and the licensed included backend drivers will generate the appropriate xml snippets required to add them to the configuration file.

<tom:environment xmlns:tom="http://www.tomsolutions.com/tom/environment/1.0">
     



    <logger level="debug" format="[%5d] [%-25s] [%-15s] [%-5s] [%s]" file="tom-util.log">

    <tsdb default-repository="fame" module-directory="/usr/local/lib/tom-tsdb">
        <repository name="fame" module="mod_fame" description="FAME">
            <options>
                <formula-fourgl path="/usr/local/lib/tom-tsdb" file="formulas.pc"/>
                <db-home paths-relative-to="1">/usr/local/famedbs</db-home>
            </options>
        </repository>
        <repository name="atlas" module="mod_adb" description="AtlasDB">
            <options>
                





                <cache-size>104857600</cache-size>
                <transactions use="1" write-ahead-logging="0">
                <datastore chunk-size="2048" page-size="32768">
                



                <db-home mode="666" paths-relative-to="1">/usr/local/adb/</db-home>
            </options>
        </repository>
        <repository name="Scylla" module="mod_cassandra" description="Scylla">
            <options>
                <contact-points>172.17.0.2,172.17.0.3</contact-points>
                <keyspace name="atlas" replication-class="SimpleStrategy" replication-config="'replication_factor': 1">
                <datastore table-name-prefix="tsdb">
            </options>
        </repository>
    </tsdb>
</tom:environment>

Examples

Example 1: Opening a database and wildcarding over all the time/sparse/scalar objects.

The following is a simple example of wildcarding. It is independent of the underlying database technology, supporting full PERL-style regex searches.

In the next examples, the #include statements and namespace modifications will be taken for granted.

#include "tom-tsdb/engine.hpp"
#include <tom-tsdb/session.hpp>
#include <tom-tsdb/connection.hpp>
#include <tom-tsdb/datastore.hpp>
#include <tom-tsdb/time_series.hpp>
#include <iostream>
#include <cstdio>
#include <time.h>

// pull in std namespace and alias tom::[tsdb|calendars]
using namespace std;
namespace db = tom::tsdb;
namespace cal = tom::calendars;

int
main(int argc, char * argv[])
{
    /* run the program passing in
         1: repository  (References the name attribute of the repository defined in tom-environment.xml)
         2: db          (The name of the corresponding DB that lives in the given repository)
         3: pattern     (A Regex patern to search for, i.e., .*.CLOSE)
    if ( argc < 4 ){ cout << "USAGE: <repository> <db> <pattern>\n"; return -1; }

    /* If the program was run using the above configuration file, it might look something like
       #>./wildcard fame na-pricing '.*.CLOSE'
       This will open the fame db named na-pricing, and wildcard all objects that have a name ending in
       '.CLOSE'
     */
    db::session & ses = db::engine::Instance().get_session(argv[1]);
    db::connection & con = ses.get_connection();
    db::datastore & ds = con.get_datastore(argv[2], db::access_mode::read_write());

    // Here we create the wildcard regex, loop over it, and print the name of each matching object
    db::datastore_match & match = ds.regex_name_search( argv[3] );
    while ( match.next() )
    {
      cout << match.name() << endl;
    }

    // It's always good to clean up and close the datastore
    ds.close();
}

Example 2: Get a time-series object from a datastore.

The following is a simple example of retrieving a time-series. It is independent of the underlying database technology. It shows a quick way to print the values as well as the simple technique to iterate over the values.


int
main(int argc, char * argv[])
{
    /* run the program passing in
         1: repository  (References the name attribute of the repository defined in tom-environment.xml)
         2: db          (The name of the corresponding DB that lives in the given repository)
         3: ts-name     (The name of the time-series object, i.e., IBM.CLOSE)
    if ( argc < 4 ){ cout << "USAGE: <repository> <db> <ts-name>\n"; return -1; }

    /* If the program was run using the above configuration file, it might look something like
       #>./get_ts fame na-pricing 'IBM.CLOSE'
       This will open the fame db named na-pricing, and retrieve IBM.CLOSE
     */
    db::session & ses = db::engine::Instance().get_session(argv[1]);
    db::connection & con = ses.get_connection();
    db::datastore & ds = con.get_datastore(argv[2], db::access_mode::read_write());
    db::time_series & ts = ds.get_time_series( argv[3] );

    // this will print the time-series out using the std::cout ostream
    cout << ts << endl;


    // internally, dates are just unsigned long ints and this is the most convenient to deal
    // with calendaring in CoreAPI.
    // A calendar object can convert from cal::date to cal::date_int_type like a smart functor
    // You can see how we get the time-series calendar, first date_int_type and iterate
    // This is the most effecient method of date iteration
    // NOTE the call to cal(start) passing in a date_int_type. This converts it to a date for printing.
    cal::date_int_type start = ts.get_first_date_int();
    cal::calendar &cal = ts.get_calendar();
    for( ; start <= ts.get_last_date_int(); ++start )
    {
        cout << cal(start) << "\t" << ts[start] << endl;
    }

    // It easy to create dates that are more humanly readable
    // NOTE how easy it is to make a date, and index into the time-series the same
    //      way you do with cal::date_int_type
    cal::date d(2017,03,20);
    cout << d << "\t" << ts[d] << endl;

    // It's always good to clean up and close the datastore
    ds.close();
}

Example 3: Migrate time-series between different database engines.

The following is a simple example of retrieving a time-series from one datastore and saving it to a different datastore. It is independent of the underlying database technology and allows for rapid migration of data between heterogeneous systems seamlessly.


 int
 main(int argc, char * argv[])
 {
    /* run the program passing in
         1: repository  (References the name attribute of the repository defined in tom-environment.xml)
         2: db          (The name of the corresponding DB that lives in the given repository)
         3: ts-name     (The name of the time-series object, i.e., IBM.CLOSE)
    if ( argc < 6 ){ cout << "USAGE: <repository1> <db1> <ts-name> <repository2> <db2>\n"; return -1; }

    /* If the program was run using the above configuration file, it might look something like
       #>./copy_ts fame na-pricing 'IBM.CLOSE' atlas testdb
       This will open the fame db named na-pricing, and retrieve IBM.CLOSE, and store a copy into
       the atlas db named testdb using the same name.
     */

    // open the fame database and get the time-series
    db::session & ses1 = db::engine::Instance().get_session(argv[1]);
    db::connection & con1 = ses1.get_connection();
    db::datastore & ds1 = con1.get_datastore(argv[2], db::access_mode::read_write());
    db::time_series & ts = ds.get_time_series( argv[3] );

    // open the atlas database
    db::session & ses2 = db::engine::Instance().get_session(argv[4]);
    db::connection & con2 = ses2.get_connection();
    db::datastore & ds2 = con2.get_datastore(argv[5], db::access_mode::read_write());

    // the time-series from FAME is now copied over into the AtlasDB
    ds2.save_time_series( ts );

    // It's always good to clean up and close the datastore
    ds1.close();
    ds2.close();
 }