I am dreaming of a Python library which abstracts the file handling of my application.

The application should run in two different configurations:

  1. No storage server. All file operations get done on the local disk.
  2. With storage server. All file operations should get done via s3.

I would like to do separation of concerns.

The application code should not care which configuration gets used. Choosing the right configuration (with or without storage server) gets done via configuration management.

I don't need all file operations which I can do via os and os.path. I just need all operations which can be done via s3.

Other required features:

  • Open source: BSD or LGPL, not GPL
  • Support for Linux. Other operating systems are not important in this context.

Distinction / out off scope

I don't want all file operations (like os.walk()). I just need the fundamental storage APIs of s3, but without a running storage server.

2 Answers 2


I suggest you give Siilo a try.

It is a storage abstraction layer for Python.

Siilo gives you a base API with operations like open and delete, that you can use without worrying about what is being used below.

It supports:

Minus points:

  • It seems that very few operations are implemented. If I understand the code correctly, if you need write you will have to implement it yourself (in that case please send a pull request)
  • No documentation nor code samples
  • Community is small and very quiet

Plus points:

  • Open source (MIT)
  • Very lightweight

There is a Python library to mock out the boto (s3) library:


Moto is a library that allows your python tests to easily mock out the boto library

It can run in standalone server mode, if you want to access it from a different programming language

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.