Integrating Poetry in Python with version control systems

Version control systems, like Git, are crucial components in today’s software application development processes. They assist in controlling modification to the project’s source code, working with other developers, and, reviewing the history of the project. In this tutorial, crucial steps of Poetry’s integration with the project’s version control are discussed to improve the Python project’s development environment. This article will demonstrate recommended approaches and practical methods for using Poetry with a VCS, specifically Git.

Poetry

This is a tool that enables the management of dependencies and packaging in Python projects. It uses pyproject.toml for project configuration and a poetry.lock file to lock dependency versions, ensuring reproducible environments.

Version Control

Git can be used as a tool to keep track of changes in the code, collaborate with other developers and have more sophisticated versioning of a project. When Poetry is integrated with a VCS like Git it ensures that all developers are using identical dependencies and project configurations making the efficient management of them inherent in the development process.

The first step here would be to create a new Python project with Poetry and then create a new git repository.

Create a new Poetry project:

poetry new my_project
cd my_project

Initialize a Git repository:

git init

Add project files to Git:

git add .

Commit the initial project setup:

git commit -m "Initial commit"

Explanation

• poetry new my_project creates a new project with a basic structure, including pyproject.toml and a directory for your project code.
• Git commands for creating version control are as follows:
• git init creates a new Git repository in the project directory.
• git add . configures all files to be ready for the first commit.
• git commit -m “Initial commit” commits the staged files with a message describing the commit.

Including pyproject. toml

The pyproject.toml file is the central configuration file for Poetry projects. It is something that includes your project and its requirement or other project on which it depends. This file should always remain under version control so that all the team members as well as the CI/CD pipelines rely upon the same project.

Including poetry.lock

The poetry. Lock file freezes the versions of your dependencies. With this file included into the version control, everyone will use the specified dependencies, avoiding the situation when some of the participants employed different versions of the required tools. When updating from previous versions of this file, ensure that a backup of this file exists in the desired control system path.

.gitignore Configuration

This means poetry is a global tool that creates a virtual setting for your project. It might be quite useful rarely and usually, you do not want to include this virtual environment in your version control system. Configure your . gitignore file to exclude it:

1. Byte-compiled / optimized / DLL files

Python generates bytecode files with extensions like .pyc, .pyo, and .pyd in the __pycache__ directory. These files are specific to the machine they were generated on and do not need to be versioned.

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

2. Virtual environment directory

Virtual environments are used to manage project-specific dependencies, and their contents are generated based on pyproject.toml and poetry.lock, so they don’t need to be tracked.

# Virtual environment directory
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

3. Distribution / packaging

These directories and files are related to Python package distribution. They are generated during the build and packaging process and do not need to be included in version control.

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg

4. Poetry virtual environment

hese lines should not be included in the .gitignore file. Both pyproject.toml and poetry.lock should be versioned because they define the project dependencies and ensure reproducibility.

# Poetry virtual environment
poetry.lock
pyproject.toml

5. PyInstaller

PyInstaller generates these files when creating executable applications. They contain metadata and configuration specific to the build process and should not be included in version control.

# PyInstaller
*.manifest
*.spec

If you have an existing Git project and want to integrate Poetry, follow these steps:

Navigate to your project directory:

cd my_existing_project

Initialize Poetry:

poetry init sets up a new pyproject.toml file in your existing project.

poetry init

Follow the prompts to set up your pyproject.toml file.

Install dependencies:

If you already have a requirements.txt file, you can convert it to pyproject.toml, poetry add $(cat requirements.txt) converts your existing dependencies to Poetry’s format.:

poetry add $(cat requirements.txt)

Add and commit changes:

git add and git commit stage and commit the changes to version control.

git add pyproject.toml poetry.lock
git commit -m "Integrate Poetry for dependency management"

Usually when dealing with a project that people contribute to individually but work in a team it is important that all contributors are in the same environment in terms of the dependencies for the project. Here are some best practices:

Installing Dependencies

After cloning the repository, team members should run:

poetry install

This command installs the dependencies specified in pyproject.toml and locks the versions from poetry.lock.

Adding New Dependencies

When adding new dependencies, use:

poetry add <package_name>

Afterwards, stage the changes in pyproject if you want to rename your package. toml and poetry. lock:

git add pyproject.toml poetry.lock
git commit -m "Add <package_name>"

Updating Dependencies

To update dependencies, use:

poetry update

Then, commit the updated poetry.

If new poetry has been written, commit all the changes and commit the updated poetry. lock file:

git add poetry.lock
git commit -m "Update dependencies"

Using poetry to master poetry in the CI/CD pipelines means that the right dependencies are present from the start, and your project compiles as expected.

Example CI Configuration for Pull Requests (GitHub Actions)

Create a .github/workflows/ci.yml file with the following content:

name: CI

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: 3.8

      - name: Install Poetry
        run: curl -sSL https://install.python-poetry.org | python3 -

      - name: Install dependencies
        run: poetry install

      - name: Run tests
        run: poetry run pytest

Explanation

• actions/checkout@v2: Checks out your repository.
• actions/setup-python@v2: Sets up a Python environment.
• Install Poetry: Installs Poetry in the CI environment.
• Install dependencies: Installs project dependencies using Poetry.
• Run tests: Runs your tests using Poetry.

The addition of Poetry with other version control systems such as Git brings about a consistent and repeatable development environment. Hence, by adhering to various guidelines like incorporating pyproject. toml and poetry.lock in version control, configuring .gitignore appropriately, and when properly creating and integrating CI/CD pipelines, the development process can be made simpler and the efficiency of a team increased.

Should I include poetry.lock in version control?

Yes, you should include poetry.lock in version control. It guarantees that every member of the team, as well as all scripts defined by CI/CD pipelines, employ the same versions of the dependencies, which in turn helps to create a coherent and manageable environment.

How do I handle changes to dependencies?

Use poetry add <package_name> to add new dependencies and poetry update to update existing ones. Commit the changes to pyproject.toml and poetry.lock to version control.

What should I add to .gitignore?

Among files to be excluded are those of the virtual environment directory where your code is being run (e. g., . venv), build artifacts, and other temporary files. Ensure that pyproject. toml and poetry. lock are not ignored.

How do I set up Poetry in a CI/CD pipeline?

Ensure you get Poetry up and running in your CI/CD environment, use poetry install to install all your dependencies and poetry run to execute the tests or build some commands. For more details, refer to the example configuration on the GitHub section for Actions.

Can I convert an existing requirements.txt file to Poetry?

Yes, an existing systems can be converted from one type to another without any problem. when you want to add the requirements. txt file to the current project simply run the command poetry add $(cat requirements. txt). This command will create a pyproject.toml file and populate it with the dependencies that you have specified.