Engines

Engines are like core components that allow some piece of software to run. In Everysk, engines serve their different purposes. We have the expression engine, the cryptography engine, the compliance engine, and so on. All engines provide a set of methods for interacting with them.

flowchart TB
    classDef orangeBorder fill:#1A1A1A,color:#E2E2E2,stroke:#FF9933,stroke-width:2px;

    EnginesDir(["Engines"]):::orangeBorder
    ComplianceDir(["Compliance"]):::orangeBorder
    CryptographyDir(["Cryptography"]):::orangeBorder
    ExpressionDir(["Expression"]):::orangeBorder
    MarketDataDir(["MarketData"]):::orangeBorder
    CacheDir(["Cache"]):::orangeBorder
    LockDir(["Lock"]):::orangeBorder
    EnginesDir --> ComplianceDir
    EnginesDir --> CryptographyDir
    EnginesDir --> ExpressionDir
    EnginesDir --> MarketDataDir
    EnginesDir --> CacheDir
    EnginesDir --> LockDir

Cache¶

The UserCache class is a helper that makes it easier to work with cache storage in Everysk applications. Built on top of BaseSDK, it provides ready-to-use methods for common tasks like saving data, retrieving it, removing items, and updating values in the cache.

from everysk.sdk.engines import UserCache

Example¶

Here is an example where we use the Market Data search method for performing a search inside the database, then we store the result in cache to be later retrieve if we make the same request:

from everysk.sdk.engines import MarketData

cache = UserCache()

cache_key = 'raw_sector_query'

cached_result = cache.get(cache_key)

if cached_result:
    return cached_result

market_data = MarketData()

result = market_data.search(
    [["raw_sector", "=", "Technology"]],
    fields=["instrument_class", "name", "gics_sector"],
    limit=1,
)

cache.set(cache_key, result, timeout=600)

return result

This example shows how to use UserCache to avoid running the same database query multiple times. It first looks for the result in the cache using a key (raw_sector_query). If the data is already cached, it returns it immediately. If not, it runs the MarketData.search query, saves the result in the cache for 600 seconds (or 10 minutes), and then returns it.

Combining the features of Lock and Cache¶

from everysk.sdk.engines import MarketData, UserLock

cache = UserCache()
cache_key = 'raw_sector_query'

# Create a distributed lock with a 10-second timeout
lock = UserLock(name='sector_query_lock', timeout=10)

cached_result = cache.get(cache_key)

if cached_result:
    return cached_result

try:
    lock.acquire()  # Acquire lock before performing expensive operation

    # Double check the cache in case another process already set it
    cached_result = cache.get(cache_key)
    if cached_result:
        return cached_result

    market_data = MarketData()
    result = market_data.search(
        [["raw_sector", "=", "Technology"]],
        fields=["instrument_class", "name", "gics_sector"],
        limit=1,
    )

    cache.set(cache_key, result, timeout=600)
    return result

finally:
    lock.release()  # Always release the lock

This example ensures that only one process at a time can run the MarketData.search and update the cache by using a UserLock. The lock is acquired before the expensive operation, and the cache is checked again inside the lock to avoid duplicate work if another process already populated it. Finally, the lock is always released in a finally block, guaranteeing proper cleanup even if an error occurs.

Compliance¶

The compliance engine applies its set of methods in three different ways: Rule applicability, filtering securities, and computing value. The applicability of the rule is determined by a logical expression that uses as arguments the characteristics of a value.

from everysk.sdk.engines.compliance import Compliance

Check¶

The check method allows the verification of compliance rules against some given rules. The method receives the following parameters:

rules: a list of rules to be checked.
datastore: the data to be checked against the rules.
metadata: The metadata to be used for checking, defaults to None.

The method returns a dictionary with a key containing the boolean value of the compliance check, in other words True if the data is compliant with the rules, False otherwise.

rules = [{'rule1': 'rule1'}, {'rule2': 'rule2'}]
datastore = [{'data': 'data'}, {'data2': 'data2'}]

Compliance.check(rules, datastore)
{'compliant': True}

Cryptography¶

The Cryptography engine provides a set of methods for generating random and unique IDs. These methods below can be used alongside with the entities to generate unique IDs for the objects.

Generate a random ID¶

The generate_random_id() method is used, as the name suggests, to generate a random ID.

The method takes an integer argument length which is the length of the ID to be generated.

And another optional argument characters which is a string of characters that will be used to generate the ID. Currently, the default value is alphanumeric characters.

from everysk.sdk.engines.cryptography import generate_random_id

generate_random_id(length=10)
'IG1aaDfZ9s'

generate_random_id(length=10, characters='1234567890')
'4918487470'

Generate a unique ID¶

The generate_unique_id() takes no arguments and it is used to generate a unique ID with a fixed length of 32 characters.

from everysk.sdk.engines.cryptography import generate_unique_id

generate_unique_id()
'ebaba8774bcf4063b9832b721fb3a2e0'

Generate a short ID¶

Finally, we have the generate_short_random_id() method which is similar to the previous one, but it generates a shorter ID with a fixed length of 8 characters.

from everysk.sdk.engines.cryptography import generate_short_random_id

generate_short_random_id()
'X5r33Hmw'

Expression¶

The expression engine breaks down and interprets expressions based on a predefined set of grammatical rules. Expressions can include variables, operators, function calls, and literal values.

Below we have the import statement for the Expression class.

from everysk.sdk.engines import Expression
expression_engine = Expression()

Get Tokens¶

The get_tokens method returns the elements used in the expression. The method takes two arguments:

expression (str): The expression to extract the elements from.
data_types (Tuple[str]): The data types of the elements in the expression.

expression_engine.get_tokens('a + b')
frozenset({'a', 'b'})

Solve¶

The solve method evaluates the expression using the provided data. The method takes two arguments:

expression (str): The expression to evaluate.
user_args (dict): The data to use in the expression.

Each key inside the user_args dictionary will be used as a value to solve the expression operation.

expression_engine.solve('a + b', {'a': 1, 'b': 2})
3

Another complex example where we have a dictionary of funds that will be used as user_args to solve an expression:

user_args = {
    "fund_class": "FIRF",
    "tax_regime": "LONG TERM",
    "target_market": "PROFESSIONAL INVESTORS",
    "nlv": 100000
}
expression = 'fund_class == "FIRF"'
expression_engine.solve(expression, user_args)
True

Lock¶

The UserLock class allows you to manage distributed locks, ensuring that only one process at a time can access a shared resource.

Below we have the import statement

from everysk.sdk.engines import UserLock

Example¶

In this example, we create a lock named my_lock with a 10-second timeout. The process acquires the lock, performs some protected work, and then releases the lock.

lock = UserLock(name='my_lock', timeout=10)

try:
    lock.acquire()
    print("Lock acquired, doing something important...")

finally:
    lock.release()

This pattern should be used whenever you need to ensure exclusive access to a resource.

Failure Case¶

If you attempt to release a lock after the timeout, the system will raise an error. This prevents accidental misuse of the lock mechanism.

lock = UserLock(name='my_lock', timeout=10)

try:
    lock.acquire()
    sleep(11)

finally:
    lock.release()

This code above will raise a LockNotOwnedError since the timeout already exceeded.

Market Data¶

The MarketData engine provides a couple of methods for search data based on a given condition, limit, or date. Also with the possibility of getting historical and security data.

from everysk.sdk.engines.market_data import MarketData
market_data = MarketData()

Search method¶

The search method allows the searching of assets based on a given condition, limit, or date. The method receives the following parameters:

conditions: Each condition is a list or tuple with a (field, operator, value) structure.
fields: A list of fields to be returned.
order_by: The field to order the results.
limit: The maximum number of records to be returned.
date: The date to be used in the search.

market_data.search([
    ['everysk_symbol', '=', 'AAPL']
],

fields=['instrument_class', 'name', 'gics_sector'],
date='20250129',
limit=10
)

Below there are all the columns that we can search for in the conditions fields, they are all database columns inside the PostgreSQL.

everysk_id
historical_data
everysk_symbol
country_of_risk
currency
error_text
error_type
exchange
gics_sector
extra_data
instrument_class
isin
last_prices
mkt_cap
name
security_class
updated_at
vendor_symbol
volume
tsv_search

Understanding the `extra_data` column¶

This column serves as a way to store extra attributes for a specific security, because of the way that is implemented we can run a direct search for these attributes, look at the example below:

market_data.search(
    [["raw_sector", "=", "Technology"]],
    fields=["instrument_class", "name", "gics_sector"],
    limit=1,
)

[
    {
        "everysk_symbol": "000660:XKRX",
        "name": "SK Hynix Inc",
        "instrument_class": "Equity",
        "everysk_id": "EQTY:000660:XKRX",
        "gics_sector": "Information Technology",
    }
]

The raw_sector is an attribute inside the extra_data JSON, but we can still run a search just like the other main columns.

Note that the search method has a 14400 seconds cache for each different request.

Using the `date` argument¶

The date parameter defines the reference date used for querying the dataset in a bitemporal model. When provided, the query returns the state of the data as of that specific date. When omitted, the query defaults to the latest dataset, which is a copy of the most recent data available.

market_data.search(
    [
        ["instrument_class", "=", "FXSpot"],
        ["everysk_symbol", "in", ["KWDUSD", "AEDUSD"]],
    ],
    fields=["last_prices"],
    date="20250915",
    limit=2,
)

[
    {
        "everysk_id": "FXSP_CURR:AED:USD",
        "last_prices": {
            "low": 0.2723,
            "date": "2025-09-08",
            "high": 0.2723,
            "open": 0.2723,
            "close": 0.2723,
            "volume": 0.0,
            "average": 0.2723,
            "adjusted_close": 0.2723,
        },
        "everysk_symbol": "AEDUSD",
    },
    {
        "everysk_id": "FXSP_CURR:KWD:USD",
        "last_prices": {
            "low": 3.2717,
            "date": "2025-09-08",
            "high": 3.2749,
            "open": 3.2748,
            "close": 3.2717,
            "volume": 0.0,
            "average": 3.2717,
            "adjusted_close": 3.2717,
        },
        "everysk_symbol": "KWDUSD",
    }
]

Below, let's search without the date argument to contextualize:

market_data.search(
    [
        ["instrument_class", "=", "FXSpot"],
        ["everysk_symbol", "in", ["KWDUSD", "AEDUSD"]],
    ],
    fields=["last_prices"],
    limit=2,
)

[
    {
        "everysk_id": "FXSP_CURR:AED:USD",
        "last_prices": {
            "low": 0.2723,
            "date": "2025-09-08",
            "high": 0.2723,
            "open": 0.2723,
            "close": 0.2723,
            "volume": 0.0,
            "average": 0.2723,
            "adjusted_close": 0.2723,
        },
        "everysk_symbol": "AEDUSD",
    },
    {
        "everysk_id": "FXSP_CURR:KWD:USD",
        "last_prices": {
            "low": 3.2717,
            "date": "2025-09-08",
            "high": 3.2749,
            "open": 3.2748,
            "close": 3.2717,
            "volume": 0.0,
            "average": 3.2717,
            "adjusted_close": 3.2717,
        },
        "everysk_symbol": "KWDUSD",
    }
]

Get Historical¶

The get_historical method allows the retrieval of historical data for a given asset. The method receives the following parameters:

date: The date to search for inside the database.
start_date: When the values for the historical should start
end_date: When the values for the historical should end
ticker_list: The list of securities to search the historical
ticker_type: The type of the securites, defaults to 'everysk_id' or 'everysk_symbol'
projection: The prices for the security. close, high, low, average, or adjusted_close.

In the example below, we retrieve the close and low prices for Apple’s security between September 1, 2025, and September 16, 2025.

market_data.get_historical(
    date='20250915',
    start_date='20250901',
    end_date='20250916',
    ticker_list=['AAPL'],
    ticker_type='everysk_symbol',
    projection=['close', 'low']
)

{
    "AAPL": {
        "everysk_symbol": "AAPL",
        "everysk_id": "EQTY:AAPL:XNAS",
        "status": "OK",
        "as_of": "2025-09-15",
        "historical_data": {
            "index": [
                "2025-09-02",
                "2025-09-03",
                "2025-09-04",
                "2025-09-05",
                "2025-09-08",
                "2025-09-09",
                "2025-09-10",
            ],
            "columns": ["low", "close"],
            "data": [
                [229.72, 229.72],
                [238.47, 238.47],
                [239.78, 239.78],
                [239.69, 239.69],
                [237.88, 237.88],
                [234.35, 234.35],
                [226.79, 226.79],
            ],
            "index_names": ["date"],
            "column_names": [None],
        },
    }
}

Engines

Cache¶

Example¶

Combining the features of Lock and Cache¶

Compliance¶

Check¶

Cryptography¶

Generate a random ID¶

Generate a unique ID¶

Generate a short ID¶

Expression¶

Get Tokens¶

Solve¶

Lock¶

Example¶

Failure Case¶

Market Data¶

Search method¶

Understanding the extra_data column¶

Using the date argument¶

Get Historical¶

Understanding the `extra_data` column¶

Using the `date` argument¶