10 minute read



As pylint has too many options, it’s recommended to use the pylint config file:

# file ~/.pylintrc, can be generated by pylint --generate-rcfile


    C0116, # Missing function or method docstring (missing-function-docstring)
    W1203, # Use lazy % formatting in logging functions (logging-fstring-interpolation)

max-line-length = 88

# List of note tags to take in consideration, separated by a comma.


# List of additional names supposed to be defined in builtins. Remember that
# you should avoid defining new builtins when possible.

But we can also ignore some warnings directly in the pylint command:

pylint . -j 0 --disable=C0116,W1203

To show all the inline ignored pylint alerts: pylint --enable=suppressed-message

Ignore Unused Argument given a Function Name Expression

Use dummy variable to ignore the Pylint warning on unused-argument.


# ignore W503 because of black format. BTW, flake8 also has W504 which is in contrary to W503.
# ignore E501, line too long because we have the same check at Pylint side already.
flake8 . \
  --exclude=venv \
  --extend-ignore=E203,E501,W503, \
  --max-complexity=7 \
  --show-source \
  --statistics \
  --count \

flake8 [a_file_path]

To show all the inline ignored flake8 alerts: flake8 --disable-noqa || true

There’s a very nice flake8 plugin called flake8-cognitive-complexity which checks the Cognitive Complexity in addition to the Cyclomatic Complexity provided by flake8 out of the box. We dont need to add extra parameter to use the Cognitive Complexity in flake8, it’s set to --max-cognitive-complexity=7 by default once the plugin is installed. By the way, Sonar sets the Cognitive Complexity threshold to 15 by default.

To fix imported but not used error in __init__.py file, could by all attribute (the most elegant) or by –per-file-ignores.


The bandit config file format is not well documented, I passed a lot of time to test the config.

$ cat .bandit
# https://github.com/PyCQA/bandit/issues/400
  - "./venv/*"

# https://github.com/PyCQA/bandit/pull/633
    - "*/*_test.py"
    - "*/test_*.py"
# without specifying -c ./bandit, it doesn't work
$ bandit . -r -c ./.bandit


ossaudit uses Sonatype OSS Index to audit Python packages for known vulnerabilities.

It can check installed packages and/or packages specified in dependency files. The following formats are supported with dparse:

  • PIP requirement files
  • Pipfile
  • Pipfile.lock
  • tox.ini
  • conda.yml
# check installed packages and packages listed in two requirements files
$ ossaudit --installed --file requirements.txt --file requirements-dev.txt
Found 0 vulnerabilities in 214 packages

Github has already provided, free of charge, the vulnerable dependencies alert.


For projects having sqlalchemy, we often install the sqlalchemy-stubs plugin as sqlalchemy uses some dynamic classes.

And also django-stubs, pandas-stubs, types-setuptools, types-requests etc.

mypy config file:

ignore_missing_imports = True # We recommend using this approach only as a last resort: it's equivalent to adding a # type: ignore to all unresolved imports in your codebase.
plugins = sqlmypy # sqlalchemy-stubs
exclude = (?x)(
    | ^build

running mypy:

mypy .
mypy . --exclude [a regular expression that matches file path]
mypy . --exclude venv[//] # exclude venv folder under the root

When using mypy, it would be better to use mypy against to all files in the project, but ont some of them,

ignore lint error in one line

linter ignore in one line
pylint (2 spaces)# pylint: disable={errorIdentifier}
flake8 (2 spaces)# noqa: {errorIdentifier}
bandit (2 spaces)# nosec
mypy (2 spaces)# type: ignore
multiple linters (2 spaces)# type: ignore # noqa: {errorIdentifier} # pylint: disable={errorIdentifier}

To ignore Pylint within a code block

# https://stackoverflow.com/a/48836605/5095636
import sys

# pylint: disable=wrong-import-position
from x import (  # noqa: E402
from y import c  # noqa: E402

# pylint: enable=wrong-import-position



isort . --profile=black --virtual-env=venv --recursive --check-only
isort . --profile=black --virtual-env=venv --recursive
isort [a_file_path]

Be very careful with isort, it’s not uncompromising, especially for some codes that dynamically import some modules inside a function instead of from the beginning of a file. People use often this to avoid circular import problem. Always run the tests after the isort.


black . --check
black .
black [a_file_path]

Using black with other tools: https://black.readthedocs.io/en/stable/guides/using_black_with_other_tools.html


Just my 2 cents, try the errorlens extension in VSCode, it will lint all the warnings/errors on live when coding, it’s really cool.

And don’t forget to install the official SonarLint extension, it will give you extra lint. It eats a lot of memory with its java processes nevertheless.

  "isort.args": [
  "python.formatting.provider": "none",
  "[python]": {
    "editor.defaultFormatter": "ms-python.black-formatter",
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {
      // "source.organizeImports": true
  "python.linting.banditEnabled": true,
  "python.linting.banditArgs": [
  "python.linting.ignorePatterns": [
  "python.linting.mypyEnabled": true,
  "python.linting.mypyArgs": [
  "sonarlint.connectedMode.connections.sonarqube": [
      "serverUrl": "https://sonar.xxx",
      "connectionId": "sonar.xxx"
  "[json]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode",
    // "editor.defaultFormatter": "esbenp.prettier-vscode",
    "editor.formatOnSave": true
  "[jsonc]": {
    "editor.defaultFormatter": "vscode.json-language-features"


pyproject.toml is the new standard in Python introduced by PEP 518 (2016) for build system requirements, PEP 621 (2020) for project metadata, and PEP 660 (2021) for wheel based editable installs.

It’s fun to know why Python authority chose this name, and very interesting to understand their POV of different file formats :smile:.

All the the major tools (setuptools, pip-tools, poetry) support this new standard, and the repo awesome-pyproject maintains a list of Python tools which are compatible to pyproject.toml.

We cannot officially declare flake8 config in pyproject.toml.

Hereunder an example of its content for the lint part.

profile = "black"

ignore_missing_imports = true
warn_return_any = true
warn_unused_configs = true
show_error_codes = true
# disallow_untyped_defs = true
# strict = true
exclude = [
    "^venv/", # we don't need to exclude `.venv` in mypy as hidden folders are excluded by default

# we dont need to exclude `.venv` in bandit as it uses wildcast here
exclude_dirs = ["venv", "_local_test"]
skips = ["*/*_test.py", "*/test_*.py"]

# ! type to use pyspark-stubs
# extension-pkg-allow-list = ["pyspark"]
# ignored-modules = ["pyspark"]
jobs = 0
# [tool.pylint.typecheck]
# # ! type to use pyspark-stubs
# generated-members = ["pyspark.sql.functions"]
good-names = [
  "df"  # for dataframe
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid defining new builtins when possible.
additional-builtins = ["spark"]
[tool.pylint."messages control"]
disable = [
notes = ["FIXME"]
max-line-length = 88
expected-line-ending-format = "LF"

# the default doesn't ignore comment line with words between `#` and `http` like:
# the url is https://pylint.pycqa.org/en/latest/user_guide/configuration/all-options.html#ignore-long-lines
ignore-long-lines = "^\\s*(#)+.*<?https?://"

    -v -s
    --cov {source_folder}

Git pre-commit


“Git hook scripts are useful for identifying simple issues before submission to code review. We run our hooks on every commit to automatically point out issues in code such as missing semicolons, trailing whitespace, and debug statements. By pointing these issues out before code review, this allows a code reviewer to focus on the architecture of a change while not wasting time with trivial style nitpicks.”

pip install pre-commit
pre-commit install

# install the script along with the hook environments in one command
# https://pre-commit.com/index.html#pre-commit-install-hooks
pre-commit install --install-hooks

# Auto-update pre-commit config to the latest repos' versions.
pre-commit autoupdate

# Clean out cached pre-commit files.
pre-commit clean

# Clean unused cached repos.
pre-commit gc

# Run single check
pre-commit run black

# continuous integration
# https://pre-commit.com/index.html#usage-in-continuous-integration
pre-commit run --all-files
# check only files which have changed
pre-commit run --from-ref origin/HEAD --to-ref HEAD

# Azure pipeline example with cache

# automatically enabling pre-commit on repositories
# https://pre-commit.com/index.html#automatically-enabling-pre-commit-on-repositories
git config --global init.templateDir ~/.git-template
pre-commit init-templatedir ~/.git-template

Online examples

pylint github pre-commit-config.yaml

Create a file named .pre-commit-config.yaml to the root of your project

Although each lint has its own config to exclude some files from checking, pre-commit also has the key exclude with list value or regex to exclude file from sending to linter.

language: system means using the executables from the same environment of current Python interpreter.

When using mypy in pre-commit, it would be better run pre-commit run --all-files, mypy doesn’t work well with only diff files sent by pre-commit run --from-ref origin/${pullrequest_target_branch_name} --to-ref HEAD.

# Installation:
# pip install pre-commit
# pre-commit install
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.3.0
      - id: check-json
        exclude: devcontainer.json
      - id: check-yaml
      - id: check-toml
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: debug-statements
      - id: requirements-txt-fixer
      - id: detect-private-key
      - id: mixed-line-ending
        args: ["--fix=lf"]
      - id: check-added-large-files
      - id: no-commit-to-branch
  - repo: https://github.com/Lucas-C/pre-commit-hooks
    rev: v1.3.1
      - id: forbid-crlf
      - id: remove-crlf
      - id: forbid-tabs
      - id: remove-tabs
  - repo: https://github.com/pre-commit/mirrors-prettier
    rev: v3.0.0-alpha.1
      - id: prettier
  - repo: https://github.com/pre-commit/pygrep-hooks
    rev: v1.9.0
      - id: python-check-blanket-type-ignore
      - id: python-check-mock-methods
      - id: python-no-log-warn
      - id: python-use-type-annotations
  - repo: https://github.com/asottile/pyupgrade
    rev: v3.1.0
      - id: pyupgrade
  - repo: local
      - id: isort
        name: isort
        entry: isort
        language: system
        types: [python]
      - id: black
        name: black
        entry: black
        language: system
        types: [python]
      - id: bandit
        name: bandit
        entry: bandit
        language: system
        types: [python]
          - -c
          - pyproject.toml
      - id: pylint
        name: pylint
        entry: pylint
        language: system
        types: [python]
      - id: flake8
        name: flake8
        entry: flake8
        language: system
        types: [python]
      - id: mypy
        name: mypy
        language: system
        entry: mypy
        types: [python]
      - id: pytest
        name: pytest
        types: [python]
        entry: pytest
        language: system
        pass_filenames: false
        always_run: true

Be aware that especially in a local environment, we often use venv, in such case, it would be better to use above system level lint executables instead of below public ones, the checks will be more accurate.

# example of using online linters
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.3.0
      - id: check-json
      - id: check-yaml
      - id: check-toml
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: debug-statements
      - id: requirements-txt-fixer
      - id: detect-private-key
      - id: mixed-line-ending
        args: ['--fix=lf']
      - id: check-added-large-files
      - id: no-commit-to-branch
  - repo: https://github.com/Lucas-C/pre-commit-hooks
    rev: v1.3.1
      - id: forbid-crlf
      - id: remove-crlf
      - id: forbid-tabs
      - id: remove-tabs
  - repo: https://github.com/psf/black
    rev: 22.8.0
      - id: black
        name: "Format with Black"
  - repo: https://github.com/pycqa/isort
    rev: 5.10.1
      - id: isort
        args: ["--profile", "black"]
  - repo: https://github.com/pycqa/flake8
    rev: 5.0.4
      - id: flake8
          - flake8-bugbear
          - flake8-comprehensions
          - flake8-simplify
  - repo: https://github.com/pre-commit/mirrors-pylint
    rev: "v3.0.0a5"
      - id: pylint
  - repo: https://github.com/pre-commit/mirrors-mypy
    # it might be better to use local venv installed mypy because it has access to all the modules installed in the venv
    rev: v0.981
      - id: mypy
          # just for example
          - types-dataclasses >= 0.1.3
          - click >= 8.1.0
  - repo: https://github.com/Lucas-C/pre-commit-hooks-bandit
    rev: v1.0.5
      - id: python-bandit-vulnerability-check
          - --recursive
          - .
          - -c
          - ./.bandit
  - repo: https://github.com/pre-commit/mirrors-prettier
    rev: v2.7.1
      - id: prettier
  - repo: local
    - id: bandit
      name: local bandit
      entry: bandit
      language: python
      language_version: python3
      types: [python]

Install the git hook scripts

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit

$ pre-commit install --hook-type post-merge
pre-commit installed at .git/hooks/post-merge

$ pre-commit install --hook-type pre-merge-commit
pre-commit installed at .git/hooks/pre-merge-commit

You could also run pre-commit install --hook-type pre-push to register pre-push hooks.

Run against all the files

“it’s usually a good idea to run the hooks against all of the files when adding new hooks (usually pre-commit will only run on the changed files during git hooks)”

pre-commit run --all-files

Run for changed files only in CI

Please check also this official doc.

git fetch origin
pre-commit run --from-ref origin/${pullrequest_target_branch_name} --to-ref HEAD

When using mypy, it would be better to use mypy against to all files in the project, but not the changed one only.

Git commit

Each time we use git commit to stage some files, these files will be sent to pre-commit to be checked against to the hooks defined in .pre-commit-config.yaml.

Temporarily disabling hooks

The official doc gives the example how to disable explicitly hooks by hooks’ ids: SKIP=flake8 git commit -m "foo", but if you want to disable completely all the hooks, an easy way might be found here by using git commit --no-verify or its shortcut git commit -n. If you use pre-commit during push, you can disable pre-commit during push by git push --no-verify or git push -n.

Automatically enabling pre-commit on repositories


Usage in continuous integration


Leave a comment