tribler.core.database.store

Attributes

`BETA_DB_VERSIONS`
`CURRENT_DB_VERSION`
`MIN_BATCH_SIZE`
`MAX_BATCH_SIZE`
`POPULAR_TORRENTS_FRESHNESS_PERIOD`
`POPULAR_TORRENTS_COUNT`
`sql_create_fts_table`
`sql_add_fts_trigger_insert`
`sql_add_fts_trigger_delete`
`sql_add_fts_trigger_update`
`sql_add_torrentstate_trigger_after_insert`
`sql_add_torrentstate_trigger_after_update`
`sql_create_partial_index_torrentstate_last_check`

Classes

`ObjState`	Different states of information in the store.
`ProcessingResult`	This class is used to return results of processing of a payload by process_payload.
`MetadataStore`	Storage of metadata for channels and torrents.

Module Contents

class tribler.core.database.store.ObjState(*args, **kwds)

Bases: enum.Enum

Different states of information in the store.

UPDATED_LOCAL_VERSION

LOCAL_VERSION_NEWER

LOCAL_VERSION_SAME

NEW_OBJECT

DUPLICATE_OBJECT

class tribler.core.database.store.ProcessingResult

This class is used to return results of processing of a payload by process_payload. It includes a dictionary of the metadata (data), the database rowid, and the state of the object as indicated by the ObjState enum.

data: dict[str, Any]

obj_state: ObjState

rowid: list[int]

tribler.core.database.store.BETA_DB_VERSIONS = [0, 1, 2, 3, 4, 5]

tribler.core.database.store.CURRENT_DB_VERSION = 15

tribler.core.database.store.MIN_BATCH_SIZE = 10

tribler.core.database.store.MAX_BATCH_SIZE = 1000

tribler.core.database.store.POPULAR_TORRENTS_FRESHNESS_PERIOD = 86400

tribler.core.database.store.POPULAR_TORRENTS_COUNT = 100

tribler.core.database.store.sql_create_fts_table = Multiline-String

Show Value

"""
    CREATE VIRTUAL TABLE IF NOT EXISTS FtsIndex USING FTS5
        (title, content='ChannelNode', prefix = '2 3 4 5',
         tokenize='porter unicode61 remove_diacritics 1');"""

tribler.core.database.store.sql_add_fts_trigger_insert = Multiline-String

Show Value

"""
    CREATE TRIGGER IF NOT EXISTS fts_ai AFTER INSERT ON ChannelNode
    BEGIN
        INSERT INTO FtsIndex(rowid, title) VALUES (new.rowid, new.title);
    END;"""

tribler.core.database.store.sql_add_fts_trigger_delete = Multiline-String

Show Value

"""
    CREATE TRIGGER IF NOT EXISTS fts_ad AFTER DELETE ON ChannelNode
    BEGIN
        DELETE FROM FtsIndex WHERE rowid = old.rowid;
    END;"""

tribler.core.database.store.sql_add_fts_trigger_update = Multiline-String

Show Value

"""
    CREATE TRIGGER IF NOT EXISTS fts_au AFTER UPDATE ON ChannelNode BEGIN
        DELETE FROM FtsIndex WHERE rowid = old.rowid;
        INSERT INTO FtsIndex(rowid, title) VALUES (new.rowid, new.title);
    END;"""

tribler.core.database.store.sql_add_torrentstate_trigger_after_insert = Multiline-String

Show Value

"""
    CREATE TRIGGER IF NOT EXISTS torrentstate_ai AFTER INSERT ON TorrentState
    BEGIN
        UPDATE "TorrentState" SET has_data = (last_check > 0) WHERE rowid = new.rowid;
    END;
"""

tribler.core.database.store.sql_add_torrentstate_trigger_after_update = Multiline-String

Show Value

"""
    CREATE TRIGGER IF NOT EXISTS torrentstate_au AFTER UPDATE ON TorrentState
    BEGIN
        UPDATE "TorrentState" SET has_data = (last_check > 0) WHERE rowid = new.rowid;
    END;
"""

tribler.core.database.store.sql_create_partial_index_torrentstate_last_check = Multiline-String

Show Value

"""
    CREATE INDEX IF NOT EXISTS idx_torrentstate__last_check__partial
    ON TorrentState (last_check, seeders, leechers, self_checked)
    WHERE has_data = 1;
"""

class tribler.core.database.store.MetadataStore(db_filename: str, private_key: ipv8.keyvault.keys.PrivateKey, disable_sync: bool = False, notifier: tribler.core.notifier.Notifier | None = None, check_tables: bool = True, db_version: int = CURRENT_DB_VERSION)

Storage of metadata for channels and torrents.

notifier = None

db_path

my_key

my_public_key_bin

_logger

_shutting_down = False

batch_size = 10

reference_timedelta

sleep_on_external_thread = 0.05

db

MiscData

TrackerState

TorrentState

TorrentMetadata

fast_integrity_check(remove_broken: bool = True) → bool

Inspect the database file and return whether it was broken or not.

By default, this method also removes the given db file if it is broken. Set remove_broken to False if you want to keep the broken file.

Returns:: Whether the database file was broken (False if no errors occurred).

set_value(key: str, value: str) → None: Set a generic key to a value.

get_value(key: str, default: str | None = None) → str | None: Retrieve the value for a given key.

drop_indexes() → None: Drop the indices for this database.

get_objects_to_create() → list[pony.orm.core.Entity]: Get the objects that need to be created.

get_db_file_size() → int: Get the physical size on disk (always 0 for memory dbs).

drop_fts_triggers() → None: Drop the FTS triggers.

create_fts_triggers() → None: Create the FTS triggers.

fill_fts_index() → None: Insert the FTS indices.

create_torrentstate_triggers() → None: Create the torrent state triggers.

shutdown() → None: Disconnect the connection to the database.

async run_threaded(func: collections.abc.Callable, *args: Any, **kwargs) → Any

Run func threaded and close DB connection at the end of the execution.

Parameters:

func – the function to be executed threaded
args – args for the function call
kwargs – kwargs for the function call

Returns:

a result of the func call.

async process_compressed_mdblob_threaded(compressed_data: bytes, **kwargs) → list[ProcessingResult]: Decompress the given data in a thread and return a list of uncompressed results.

process_compressed_mdblob(compressed_data: bytes, skip_personal_metadata_payload: bool = True) → list[ProcessingResult]: Decompress the given data and return a list of uncompressed results.

process_torrent_health(health: tribler.core.torrent_checker.healthdataclasses.HealthInfo) → bool

Adds or updates information about a torrent health for the torrent with the specified infohash value.

Parameters:: health – a health info of a torrent
Returns:: True if a new TorrentState object was added

process_squashed_mdblob(chunk_data: bytes, external_thread: bool = False, health_info: list[tuple[int, int, int]] | None = None, skip_personal_metadata_payload: bool = True) → list[ProcessingResult]

Process raw concatenated payloads blob. This routine breaks the database access into smaller batches. It uses a congestion-control like algorithm to determine the optimal batch size, targeting the batch processing time value of self.reference_timedelta.

Parameters:

chunk_data – the blob itself, consists of one or more GigaChannel payloads concatenated together
external_thread – if this is set to True, we add some sleep between batches to allow other threads to get the database lock. This is an ugly workaround for Python and asynchronous programming (locking) imperfections. It only makes sense to use it when this routine runs on a non-reactor thread.
health_info – the health info to update a torrent with.
skip_personal_metadata_payload – don’t process our own torrents.

Returns:

a list of tuples of (<metadata or payload>, <action type>)

process_payload(payload: tribler.core.database.serialization.TorrentMetadataPayload, skip_personal_metadata_payload: bool = True) → list[ProcessingResult]: Write a payload to our database (if necessary).

get_num_torrents() → int: Get the number of torrents in the database.

search_keyword(query: str, origin_id: int | None = None) → pony.orm.core.Query

Search for an FTS query, potentially restricted to a given origin id.

Requires FTS5 table “FtsIndex” to be generated and populated. FTS table is maintained automatically by SQL triggers. BM25 ranking is embedded in FTS5.

This method implements REST-friendly way to get entries from the database.

Warning! For Pony magic to work, iteration variable name (e.g. ‘g’) should be the same everywhere!

Returns:: PonyORM query object corresponding to the given params.

async get_entries_threaded(**kwargs) → list[tribler.core.database.orm_bindings.torrent_metadata.TorrentMetadata]: Retrieve entries in a thread and return a list of results.

get_entries(first: int = 1, last: int | None = None, **kwargs) → list[tribler.core.database.orm_bindings.torrent_metadata.TorrentMetadata]

Get some torrents. Optionally sort the results by a specific field, or filter the channels based on a keyword/whether you are subscribed to it.

Returns:: A list of class members

get_total_count(**kwargs) → int | None: Get total count of torrents that would be returned if there would be no pagination/limits/sort.

get_entries_count(**kwargs) → int | None: Get the count of torrents that would be returned if there would be no pagination/limits.

get_max_rowid() → int: Get the highest-known row id.

fts_keyword_search_re

get_auto_complete_terms(text: str, max_terms: int) → list[str]: Get the auto-completion terms for a given query.