Search and Discovery Mechanisms

etchblok-test-api provides two primary ways to find saved content: paginated listing for browsing and full-text search for targeted discovery. Both mechanisms are orchestrated by the BookmarkService in app/services/bookmark_service.py, which acts as a facade over the repository and search index.

Paginated Listing

When you need to display a list of bookmarks in a UI, fetching every record at once is inefficient. The BookmarkService.list_bookmarks method provides a paginated interface that supports filtering by status (e.g., active, archived, or trashed).

Usage

You can request a specific page and set the number of items per page. If you provide a status, the results are filtered accordingly.

from app.services.bookmark_service import BookmarkService

service = BookmarkService()

# Get the second page of active bookmarks, 10 per page
bookmarks, total_count = service.list_bookmarks(
    page=2, 
    per_page=10, 
    status="active"
)

Internal Implementation

The BookmarkService delegates this operation to the BookmarkRepository in app/db/repository.py. The repository performs the following steps:

1. Filtering: It filters the in-memory collection based on the BookmarkStatus enum. If an invalid status string is provided, it silently ignores the filter.
2. Sorting: It sorts all matching items by their created_at timestamp in descending order (newest first).
3. Slicing: It calculates the start index using (page - 1) * per_page and returns a slice of the list.

# app/db/repository.py

def list_bookmarks(self, page: int = 1, per_page: int = 25, status: Optional[str] = None):
    items = list(self._bookmarks.values())
    if status:
        try:
            target = BookmarkStatus(status)
            items = [b for b in items if b.status == target]
        except ValueError:
            pass
    items.sort(key=lambda b: b.created_at, reverse=True)
    total = len(items)
    start = (page - 1) * per_page
    return items[start : start + per_page], total

Full-Text Search

For finding bookmarks by keyword, etchblok-test-api implements an in-memory inverted index via the SearchIndex class in app/services/search_service.py.

Usage

Call the search method with a query string. You can also specify a limit to restrict the number of results returned.

# Search for bookmarks containing "python" and "tutorial"
results = service.search("python tutorial", limit=5)

How Search Works

The SearchIndex processes queries using several steps to ensure relevant results:

1. Tokenization: The query and the bookmark content (title and description) are broken into lowercase alphanumeric tokens. Common "stop words" like "the", "and", and "is" are removed to improve relevance.
2. AND Logic: The search uses strict AND logic. Every token in your query must appear in either the title or the description for a bookmark to be included in the results.
3. Relevance Ranking: Results are ranked by how many times the query tokens appear in the bookmark's text. A bookmark where "python" appears three times will rank higher than one where it appears once.

# app/services/search_service.py

def search(self, query: str, limit: int = 20) -> List[Bookmark]:
    tokens = self._tokenize(query)
    if not tokens:
        return []

    # Intersect sets of IDs for each token (AND logic)
    candidate_ids: Set[str] = self._index.get(tokens[0], set()).copy()
    for token in tokens[1:]:
        candidate_ids &= self._index.get(token, set())

    # ... retrieve bookmarks and rank them ...
    return self._rank_results(results, tokens)[:limit]

Maintaining Index Consistency

The BookmarkService ensures that the search index and the repository remain synchronized. Whenever you create or update a bookmark, the service automatically updates the index.

• Creation: When create_bookmark is called, the new bookmark is persisted to the repository and then passed to self._search.index_bookmark(bookmark).
• Updates: When update_bookmark is called, the service re-indexes the bookmark. The SearchIndex handles this by first removing the old entries for that bookmark ID and then adding the new tokens.
• Deletion: While delete_bookmark performs a soft-delete (moving the item to the trash), the bookmark remains in the search index.

Performance and Architecture

etchblok-test-api is designed for speed with small-to-medium datasets using several architectural patterns:

• Singleton Service: BookmarkService uses the Singleton pattern to ensure that the SearchIndex and LRUCache are shared across the entire application. This prevents redundant indexing and ensures cache consistency.
• LRU Cache: The service maintains an LRUCache (defined in app/services/_cache.py) with a maximum size of 256 items. get_bookmark calls check this cache before hitting the repository.
• In-Memory Nature: Both the repository and the search index reside in memory. The SearchIndex is completely rebuilt from the repository whenever the application starts (via _init_services).

[!WARNING] Because the search index is in-memory and rebuilt on startup, it is not intended for extremely large datasets or persistent storage across restarts without a backing repository.