# python-template
[](https://gitlab.cc-asp.fraunhofer.de/babu/python-template/-/commits/main)
[](https://gitlab.cc-asp.fraunhofer.de/babu/python-template/-/commits/main)
[](https://gitlab.cc-asp.fraunhofer.de/babu/python-template/-/releases)
This is a template for a project with the following features:
- Setuptools for Python project
- Sphinx documentation
- Pytest unit tests with coverage
- Pylint code quality checks
- Type checking with mypy
- Black code formatting
- Gitlab CI/CD
## Documentation 📖
Auto generated documentation can be found _[here](http://python-template-babu-5d849e1f859ea5533ef4f19b88fd2c5db39a37f7ed.pages.fraunhofer.de/)_. :point_left:
## Setup Sphinx
Sphinx is a documentation generator that can be used to generate documentation from docstrings in the code.
**All commands are to be run from the project directory.**
<details>
<summary>Setup Sphinx for a New Project</summary>
1. Delete the existing `docs` directory if it exists.
```bash
rm -rf docs
```
1. Create Sphinx Project
We will use the `sphinx-quickstart` command to create a new Sphinx project.
The files will be created in the `docs` directory.
```bash
sphinx-quickstart docs
```
1. Edit `conf.py`
Edit the `docs/conf.py` file to add the following lines to the top of the file:
```python
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
```
Add the following lines to the `extensions` list:
```python
extensions = [
"sphinx.ext.autodoc", # for autodoc
"sphinx.ext.autosummary", # for autosummary
"sphinx.ext.viewcode", # for source code
"sphinx.ext.napoleon", # for google style docstrings
"sphinx_autodoc_typehints", # for type hints
"sphinx_copybutton", # for copy button
"sphinx-prompt", # for prompt
"recommonmark", # for markdown
]
```
Change the theme to `sphinx_rtd_theme` or another theme of your choice if you prefer:
```python
html_theme = "sphinx_rtd_theme"
```
Add the following lines to the end of the file:
```python
# generate autosummary even if no references
autosummary_generate = True
autosummary_imported_members = True
```
1. Generate Documentation
Run the following command to generate the documentation:
```bash
sphinx-apidoc -f -e -M -o docs/ sample_project_ivi/
```
The above command will generate the documentation for the `sample_project_ivi` package. There will be a `modules.rst` file in the output directory. This file will be used to generate the table of contents for the documentation.
1. Edit `index.rst`
Add modules to the table of contents by adding the following lines to the `docs/index.rst` file:
```rst
.. toctree::
:maxdepth: 2
:caption: Contents:
modules
```
1. Build Documentation
Run the following command to build the documentation:
```bash
# for html documentation
sphinx-build -b html docs docs/_build
# for pdf documentation (requires latex to be configured)
sphinx-build -M latexpdf docs docs/_build
```
The documentation will be generated in the `docs/_build` directory.
</details>
## Setuptools
<details>
<summary>Setup Setuptools for a New Project</summary>
- Use **setuptools** to package your code.
- Setuptools lets you easily download, build, install, upgrade, and uninstall Python packages.
- Setuptools can be configured using the following files:
- `setup.py`
- `setup.cfg`
- Setuptools lets you install your code using pip.
```bash
pip install git+REPO_URL
# or
pip install .
# or in editable mode
pip install -e .
```
- You can also save configurations for your project in the `setup.cfg` file.
- Sample `setup.py` file:
```python
from setuptools import setup, find_packages
setup(
name="project_name",
version="0.1.0",
description="Project description",
author="Author name",
author_email="Author email",
url="Project url",
license="License name",
# find_packages() finds all the packages in the src directory
# package_dir={"": "src"} can be used to specify the source directory
packages=find_packages(),
# package_data={"": ["data/*.txt"]} can be used to include data files
# the following command will include all txt files in the data directory
# hence hardcoded paths in the code can be avoided and the code can be made more portable
package_data={"src": ["data/*.txt"]},
# install_requires can be used to specify the dependencies of the project
# will be installed automatically when the project is installed
install_requires=[
"package1",
"package2",
],
# extras_require can be used to specify the dependencies of the project
# will not be installed automatically when the project is installed
extras_require={
"dev": [
"package3",
"package4",
],
},
# entry_points can be used to specify the command line scripts
entry_points={
"console_scripts": [
"script_name=package.module:main",
],
},
)
```
- Check the [Setuptools documentation](https://setuptools.pypa.io/) for more information.
</details>
## Unit testing with Pytest
<details>
<summary>Setup Pytest for a New Project</summary>
- Use **pytest** to write and run tests.
- Unittesting makes sure that your code works as expected.
- Pytest can automatically find and run tests in files named `test_*.py` or `*_test.py`.
- Pytest can be run using the following command:
```bash
pytest test_file.py
# or
python -m pytest test_file.py
# or to run all tests in a directory 'tests'
pytest tests/
```
- Here is a sample test file:
```python
import pytest
def test_function():
assert 1 == 1
```
- Some sample tests can be found in the [tests](https://gitlab.cc-asp.fraunhofer.de/babu/python-template/-/tree/main/tests).
- Check the [Pytest documentation](https://docs.pytest.org/) for more information.
- Use **coverage** to check the test coverage of your code.
- Coverage identifies which parts of your code are executed during testing and reports can be generated that show which parts of your code are missed by the tests.
- Coverage can be run using the following command:
```bash
coverage run -m pytest file.py
coverage report
```
- Check the [Coverage documentation](https://coverage.readthedocs.io/) for more information.
- **tox** can be used to run all the above tools in one command.
- Tox creates a virtual environment for each Python version you want to test.
- Tox can be run using the following command:
```bash
tox
```
- Configuration for tox can be specified in the following files:
- `tox.ini`
- `pyproject.toml`
- `setup.cfg`
- A sample tox configuration file:
```ini
[tox]
envlist = py{38,311} # specify the python versions to test
[testenv]
deps =
pytest
coverage
commands =
pytest
coverage run -m pytest
coverage report
```
- Check the [Tox documentation](https://tox.readthedocs.io/) for more information.
- Sample tox configuration files can be found in the [tox](https://gitlab.cc-asp.fraunhofer.de/babu/python-template/-/blob/main/tox.ini) directory.
</details>
## Code Quality Checks
<details>
<summary>Setup Code Quality Checks for a New Project</summary>
### Pylint
- Use **linters** to check your code for errors and style.
- Pylint is one of the tools that can be used to lint your code.
- [flake8](https://flake8.pycqa.org/) is another popular tool that can be used to lint your code.
- Code can be linted using the following command:
```bash
# recommended (pylint and flake8)
pylint file.py
# or
flake8 file.py
```
- Pylint can provide you with a score for your code. This score can be used to track the quality of your code and also be used to enforce a minimum score. In the template repository, the minimum score is set to 9.0. If the score is below 9.0, the CI pipeline will fail. This is done with the following command:
```bash
pylint --fail-under=9 uam
```
- It can be used to enforce a coding style, find bugs and unused code.
- Check the [Pylint documentation](https://pylint.pycqa.org/en/stable/index.html) for more information.
- [ruff](https://github.com/charliermarsh/ruff) can also be used to lint your code. ruff is based on rust is much faster than pylint and flake8.
- Example with pylint:
```python
# Example code test.py
def calculate_average(numbers):
total = sum(numbers)
number_of_items = len(numbers)
average = total / len(numbers)
print("The average is:", average)
numbers = [1, 2, 3, 4, 5]
calculate_average(numbers)
```
```bash
pylint test.py
# Output
************* Module example
test.py:10:0: C0304: Final newline missing (missing-final-newline)
test.py:1:0: C0114: Missing module docstring (missing-module-docstring)
test.py:2:0: C0116: Missing function or method docstring (missing-function-docstring)
test.py:2:22: W0621: Redefining name 'numbers' from outer scope (line 9) (redefined-outer-name)
test.py:4:4: W0612: Unused variable 'number_of_items' (unused-variable)
------------------------------------------------------------------
Your code has been rated at 2.86/10 (previous run: 4.29/10, -1.43)
```
The same suggestions can be seen in VSCode as well once the extensions are installed.
### Mypy
- Use **type hints** to specify the type of variables and arguments.
- Type hints are optional in Python, but they are recommended.
- Type hints can be very useful for documentation and debugging.
- Type hints are specified using the following syntax:
```python
variable: type
def function(argument: type) -> type:
pass
class Class:
def method(self, argument: type) -> type:
pass
```
- Check the [PEP 484](https://www.python.org/dev/peps/pep-0484/) for more information.
- Tools like [mypy](http://mypy-lang.org/) can be used to check your code for type errors.
```bash
mypy file.py
mypy src/
```
- It is a static type checker that can identify type errors in your code.
```python
def function(argument: int) -> int:
return argument + "1"
```
```bash
mypy file.py
error: Unsupported operand types for + ("int" and "str")
```
- Check the [Mypy documentation](http://mypy-lang.org/) for more information.
- Example 1
```python
# Example code test_1.py
def add_numbers(a: int, b: int) -> int:
return a + b
result = add_numbers(5, "10") # Type mismatch: 'str' cannot be added to 'int'
print(result)
```
```bash
mypy test_1.py
# Output
test.py:5: error: Argument 2 to "add_numbers" has incompatible type "str"; expected "int" [arg-type]
Found 1 error in 1 file (checked 1 source file)
```
- Example 2
```python
# Example code test_2.py
import numpy as np
def calculate_mean(data: np.ndarray) -> float:
return np.mean(data)
numbers = [1, 2, 3, 4, 5] # Incorrect type: List[int] instead of np.ndarray
mean = calculate_mean(numbers)
print(mean)
```
```bash
mypy test_2.py
# Output
test.py:8: error: Argument 1 to "calculate_mean" has incompatible type "List[int]"; expected "ndarray[Any, Any]" [arg-type]
Found 1 error in 1 file (checked 1 source file)
```
- **You need to provide type hints for your code to be checked by mypy. Else it will not be able to check the type of the variables.**
- Mypy will also check for type errors in third-party libraries.
### Black
- Use **black** to format your code.
- Black lets you format your code in a consistent way.
- Black can be run using the following command:
```bash
black --check --diff file.py # will show the changes that will be made without making them
# or
black file.py # will format the file in place
```
- It can be integrated with your editor to format your code on save.
- Check the [Black documentation](https://black.readthedocs.io/) for more information.
- Example:
```python
# before
def my_function():
x=1
y = 2
z=3
if x>y:
print("x is greater than y")
else:
print("y is greater than or equal to x")
```
<div align="center">
<font size="5">↓</font>
</div>
```python
# after
def my_function():
x = 1
y = 2
z = 3
if x > y:
print("x is greater than y")
else:
print("y is greater than or equal to x")
```
- Use **isort** to sort your imports.
- Isort does sort import in the following order:
- standard library imports
- related third party imports
- local application/library specific imports
- Isort can be run using the following command:
```bash
isort --check-only --diff file.py # will show the changes that will be made without making them
# or
isort file.py # will format the file in place
```
- It can also be integrated with your editor to sort your imports on save.
- Here is an example of how isort sorts imports:
```python
# before
import os
import sys
import django
import requests
import uam
```
<div align="center">
<font size="5">↓</font>
</div>
```python
# after
import os
import sys
import django
import requests
import uam
```
- Check the [Isort documentation](https://pycqa.github.io/isort/) for more information.
</details>