Make directory:
mkdir my-python-project
Navigate to new directory:
cd my-python-project
Learn more about command-line and the shell.
Init local Git repo:
git init
Create a Github repo and connect it:
git remote add origin https://github.com/user/repo.git
Create empty .gitignore file:
touch .gitignore
Create virtual environment:
python3 -m venv .venv
That creates a virtual environment in the .venv folder.
Project dependencies will be installed there.
Start virtual environment:
source .venv/bin/activate
Add to .gitignore:
.venv
Your project may depend on packages.
For a repeatable workflow, declare those in a requirements.txt file.
Create a requirements.txt file:
numpy
opencv-python
scikit-image
matplotlib
*You can also specify versions of those packages if needed.
Then install the requirements:
pip install -r requirements.txt
Find more packages on pypi.org.
A common way to organize a project is to have one main module, additional modules with specific functionality, and a tests folder.
A Python module is a file typically containing function or class definitions.
image_helpers.py:
import copy
import numpy
import cv2
from skimage import io
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
def get_image_pixels(image_url):
""" Returns a nested list of the pixels for the image located at image_url"""
image = io.imread(image_url)
image2 = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
pixel_data = numpy.asarray(image2).tolist()
for row in pixel_data:
for pixel in row:
pixel[0], pixel[2] = pixel[2], pixel[0]
return pixel_data
def render_pixels(pixel_data):
""" Displays the image represented by pixel_data"""
transformed_data = copy.deepcopy(pixel_data)
for row in transformed_data:
for pixel in row:
pixel[0], pixel[2] = pixel[2], pixel[0]
cv2.imwrite('rendered.jpg', numpy.array(transformed_data))
image = mpimg.imread('rendered.jpg')
plt.imshow(image)
Importing a whole module:
import image_helpers
pixel_data = image_helpers.get_image_pixels(url)
image_helpers.render_pixels(pixel_data)
Importing specific names:
from image_helpers import get_image_pixels, render_pixels
pixel_data = get_image_pixels(url)
render_pixels(pixel_data)
Importing all names:
from image_helpers import *
pixel_data = get_image_pixels(url)
render_pixels(pixel_data)
It's possible to assign an alias to imports:
import image_helpers as ih
pixel_data = ih.get_image_pixels(url)
I don't recommend aliasing a class or function name:
from image_helpers import get_image_pixels as gip
pixel_data = gip(url) # 🤮
But aliasing a package is okay if it's the convention:
import numpy as np
pixel_data = np.asarray(image2).tolist()
This command runs a code module:
python main.py
When run like that, Python sets a global variable __name__
to "main". That means you often see code at the bottom of modules like this:
if __name__ == "__main__":
# use the code in the module somehow
The code inside that condition will be executed as well, but only when the module is run directly.
Projects often distinguish between "prod" requirements (needed for the production deploy) and "dev" requirements (needed for local development only).
Add to requirements-dev.txt:
-r requirements.txt
black
pytest
coverage
pytest-cov
pre-commit
ruff
Then install the requirements:
pip install -r requirements-dev.txt
Many tools can be configured in the pyproject.toml file.
Create that file in your root folder.
A linter identifies code style issues as well as possible bugs. The most common linter is flake8 but there's a new faster linter called ruff.
Add options to pyproject.toml:
[tool.ruff]
line-length = 100
ignore = ["D203"]
show-source = true
Run ruff on a file or folder:
# Error out if there are Python syntax errors or undefined names
python3 -m ruff . --select=E9,F63,F7,F82
# Show warnings for other (stylistic) issues
python3 -m ruff .
The most popular formatter is black, which is PEP 8 compliant and fairly opinionated.
Add options to pyproject.toml:
[tool.black]
line-length = 100
target-version = ['py311']
Run black on a file or folder:
python3 -m black --verbose . tests/
⚠️ By default, black will rewrite the files. Use --check
option if you just want to be notified of issues.
The built-in testing framework for Python is unittest, but another popular framework is pytest.
Add options to pyproject.toml:
[tool.pytest.ini_options]
addopts = "-ra --cov"
testpaths = ["tests"]
pythonpath = ['.']
Run tests:
pytest
It's easy to forget to run a formatter or linter. Use pre-commit to make sure they're always run before a commit.
Create a .pre-commit-config.yaml file:
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 24.1.1
hooks:
- id: black
args: ['--config=./pyproject.toml']
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.2.1
hooks:
- id: ruff
Install the hooks:
pre-commit install
It's also helpful to use Github actions for another set of checks before merging code into main.
Create a .github/workflows/python.yaml file:
name: Python checks
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3
uses: actions/setup-python@v3
with:
python-version: 3.11
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Lint with ruff
run: |
ruff . --select=E9,F63,F7,F82
ruff . --exit-zero
- name: Check formatting with black
uses: psf/black@stable
with:
src: ". tests/"
options: "--check --verbose"
- name: Run unit tests
run: |
pytest