In-Memory Storage Design and Limitations

The etchblok-test-api uses an in-memory storage strategy for its data layer. This design choice prioritizes simplicity and rapid development for testing environments, providing a clean repository abstraction that can be swapped for a persistent database without changing the service-layer logic.

Storage Implementation

The core of the persistence layer is the BookmarkRepository class in app/db/repository.py. It manages three primary entities—bookmarks, tags, and collections—using standard Python dictionaries.

class BookmarkRepository:
    def __init__(self) -> None:
        self._bookmarks: Dict[str, Bookmark] = {}
        self._tags: Dict[str, Tag] = {}
        self._collections: Dict[str, Collection] = {}

Because these are standard dictionaries, all mutation methods like save_bookmark or delete_tag persist changes immediately to the application's memory. There is no external I/O or serialization involved during these operations.

Transactional Integrity and Atomicity

A significant limitation of the current etchblok-test-api architecture is the lack of ACID (Atomicity, Consistency, Isolation, Durability) transactions. In a traditional database, a series of related updates can be wrapped in a transaction to ensure they all succeed or fail together. In etchblok-test-api, multi-step operations are performed sequentially in the service layer, and a failure midway can leave the system in an inconsistent state.

For example, the delete_tag method in app/services/bookmark_service.py must remove a tag from the repository and also strip that tag from every bookmark that uses it:

def delete_tag(self, tag_id: str) -> bool:
    """Delete a tag and strip it from all bookmarks."""
    tag = self._repo.get_tag(tag_id)
    if not tag:
        return False
    # Multi-step operation without a transaction
    for bookmark in self._repo.get_bookmarks_with_tag(tag_id):
        bookmark.remove_tag(tag_id)
        self._repo.save_bookmark(bookmark)
        self._cache.invalidate(bookmark.id)
    self._repo.delete_tag(tag_id)
    return True

If the application crashes or an exception occurs during the loop, some bookmarks will have the tag removed while others will still retain it, and the tag itself may remain in the repository. The BookmarkRepository docstring explicitly acknowledges this: "For a real database you'd add transaction support here."

Scalability and Performance

The in-memory design introduces specific performance characteristics that do not scale linearly with data volume:

• Linear Scans: Operations like list_bookmarks do not benefit from database indexing. To filter by status or sort by date, the repository must convert the entire dictionary of bookmarks into a list and process it in memory.
• Memory Constraints: Since all data is stored in RAM, the maximum capacity of the API is strictly limited by the memory allocated to the application process.
• Sorting Overhead: Sorting occurs on every request to list_bookmarks, as shown in app/db/repository.py:

def list_bookmarks(self, page: int = 1, per_page: int = 25, status: Optional[str] = None) -> Tuple[List[Bookmark], int]:
    items = list(self._bookmarks.values())
    if status:
        # Linear filter
        items = [b for b in items if b.status == target]
    # Full sort on every call
    items.sort(key=lambda b: b.created_at, reverse=True)
    # ... pagination logic ...

Data Volatility

Data in etchblok-test-api is volatile. Because there is no persistence to disk (e.g., via SQLite or a JSON file), all bookmarks, tags, and collections are wiped whenever the application process restarts. This makes the current implementation suitable for ephemeral testing and CI/CD pipelines but unsuitable for production use where data retention is required.

Future Evolution

While the repository currently uses dictionaries, the codebase includes a blueprint for a more robust system in app/db/_connection.py. This internal module defines a _ConnectionPool and a _Connection class with stubs for begin_transaction, commit, and rollback.

The existence of these stubs suggests that etchblok-test-api is architected to eventually move away from in-memory dictionaries toward a real database driver, using the BookmarkRepository as the interface to hide these implementation details from the rest of the application.