tribler.core.database.augmenter

Attributes

logger
LOG_LEVELS	logging {50: CRITICAL/FATAL, 40: ERROR, 30: WARNING/WARN, 20: INFO, 10: DEBUG, 0: NOTSET}

Classes

AugmentedSearch

This class is responsible for "slow" search. It creates its own language, based on torrent titles, to augment

Module Contents

tribler.core.database.augmenter.logger

tribler.core.database.augmenter.LOG_LEVELS

logging {50: CRITICAL/FATAL, 40: ERROR, 30: WARNING/WARN, 20: INFO, 10: DEBUG, 0: NOTSET} SentencePieceTrainer { 3: FATAL, 2: ERROR, 1: WARNING, 0: INFO }

class tribler.core.database.augmenter.AugmentedSearch(config: tribler.tribler_config.TriblerConfigManager, notifier: tribler.core.notifier.Notifier, task_manager: ipv8.taskmanager.TaskManager)

This class is responsible for “slow” search. It creates its own language, based on torrent titles, to augment user queries.

config

max_title_length = 4192

title_window: list[str] = []

initialized = False

write_completed: asyncio.Future[None]

task_manager

model_file

title_cache_file

processor: sentencepiece.SentencePieceProcessor

on_shutdown() → None

Quickly dump our cache onto the disk.

consume_torrent_metadata(metadata: dict) → None

We found a new torrent title. Put it in the cache and start learning when we have sufficient data.

write(trained_model: bytes) → None

SentencePiece really wants to write to disk and then load it again. We need to be sure that new input was actually written to disk before reading it, or we lose data.

async study() → None

Study our most recent torrent titles and add them to the top tokens that we have already found in the past.

needs_kickstart() → bool

Check if we could use a good seeding.

to_phrases(pieces: list[str]) → list[list[str]]

What we get from SentencePiece is pieces. What we want is the pieces that make up a word, i.e., a phrase. Note that each word is separated by “▁” and may span several list elements.

augment(search: str, limit: int = 1000, offset: int = 1) → tuple[str, list[str]]

Augment the original user search string and create an SQL-injection-safe SQL query and its parameters.