Skip to content

PersistentDict cleanup; create WriteOncePersistentDict

Matt Wala requested to merge mattwala/pytools:persistent-dict-caching into master

Major changes to PersistentDict:

  • Uses a per-item-dir lock instead of a per-container-dir lock.
  • Removed info files from store().
  • Checks for cache collision on delete.
  • Added more tests.

This change also implements WriteOncePersistentDict, which has a write-once policy. WriteOncePersistentDict uses a writer lock but allows for unlocked reads, which makes reads from disk faster. It also uses an in-memory LRU cache to speed up accesses.

To use the cache, you supply a in_mem_cache_size parameter.

Both classes are backwards compatible with existing persistent dict directory structures.

This change requires a version bump.

Edited by Matt Wala

Merge request reports