2

In a project of mine I have multiple, independent software components.

At the moment, the situation looks like this:

root/
    web_scraping/
    rest_api/
    migrations/
    script.py

web_scraping/ is a python project built with scrapy. I use it to crawl a website and scrape some data that I store in a Redis queue.

rest_api/ is a Flask app + REST API built with Flask, Flask-SQLAlchemy and Flask-RESTPlus. I also use Flask-Migrate to create Alembic migration scripts to upgrade/downgrade a PostgreSQL database to a specific revision.

migrations/ contains the database migrations. Since the database could be used from other components than the Flask app, I think it's better to leave this migration repository in the root directory.

script.py is a python script thtat I use to process the items stored in Redis and insert them in PostgreSQL.

This scenario is already a bit challenging, because I need different python environments for the different components. E.g. A python 3.6 virtual environment is fine for the Flask app + REST API, but for the web scraping component I have to use a conda environment (because of scrapy and its dependency on Twisted).

I plan to add a GraphQL API built in Javascript (I still have to decide between apollo-server and graphql-yoga) and a React frontend which can consume either the REST API and/or the GraphQL API.

So in the end the entire project will look like this:

root/
    web_scraping/
    rest_api/
    migrations/
    script.py
    graphql_api/
    frontend/

My first thought was to create one repo for each software component, but I keep reading good stuff about the monorepo and decided to give it a try.

I know that for Javascript projects one could use Lerna. And for Python projects there should be Pants, as described in this article.

My question is, what to do when multiple languages (Python, Javascript) are involved?

  • Very interesting question. I have wondering that myself for our projects. Hope you can share what you ended up doing. – elpddev Jun 6 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.