Troubleshooting Python Twine Cannot Upload Package On Windows
Python has several tools to upload packages to PyPi or some private Artifactory locations. The mostly used one should be twine. Although twine is not a Python originate tool, but it’s officially recommended by Python.org.
Building the package
Just a quick callback on how to build the pacakge. We need to create a file named setup.py at the root of the app. Use another file named MANIFEST.IN to include the non-code files to the package. Don’t forget to set
A Built Distribution format introduced by PEP 427, which is intended to replace the Egg format. Wheel is currently supported by pip.
Before the build, ensure that
version key in
setup.py is well defined.
# to build a python wheel package # sdist will generate a .tar.gz file in dist/ # bdist_wheel will generate a .whl file in dist/ python setup.py sdist bdist_wheel
Upload built package to PyPi or private Artifactory.
We use twine to upload the Python packages. Before using it, we need to create a file name
There’s an example from jfrog for .pypirc.
Then, we can upload the package by:
# -r dev, dev is a repo defined in the ~/.pypirc file. 6.2.0> twine upload dist/* -r dev --cert [path_of_artifactory_site_cert_bundle_full_chain_in_pem_format_it_seems_that_no_param_to_ignore_ssl_error_with_twine]
.pypirc path error
Unfortunately, on Windows OS, you might get following error message:
6.2.0> twine upload dist/* --cert [artifactory_site_cert_full_chain_in_pem_format] -r dev InvalidConfiguration: Missing 'dev' section from the configuration file or not a complete URL in --repository-url. Maybe you have a out-dated '~/.pypirc' format? more info: https://docs.python.org/distutils/packageindex.html#pypirc
This error is too generic, one of the reasons is because twine cannot find the file
~/.pypirc, but if you check by
get-content ~/.pypirc, it exits.
The reason for this error is that if you’re on Windows, and
$env:HOME exists and doesn’t point to the same location as
~/ as per os.path.expanduser(), but Windows powershell uses
$env:HOME is not set by Windows by default. And Windows administrators often use
$env:HOME to redirect the user roaming profile.
.pypirc path error reason
Firstly, I set $env:HOME to a temp file, so it is differnet than $env:USERPROFILE
# Initially $env:HOME doesn't exist 6.2.0> Get-ChildItem env: | Out-String -st | Select-String 'userpro|home' ANDROID_SDK_HOME C:\Android HOMEDRIVE C: HOMEPATH \Users\xiang USERPROFILE C:\Users\xiang 6.2.0> $env:HOME = 'c:/temp' # now, we have $env:HOME which is different than $env:USERPROFILE 6.2.0> Get-ChildItem env: | Out-String -st | Select-String 'userpro|home' ANDROID_SDK_HOME C:\Android HOME c:/temp HOMEDRIVE C: HOMEPATH \Users\xiang USERPROFILE C:\Users\xiang
- Check ~/ in Python
In : import os In : os.path.expanduser('~/') Out: 'c:/temp/'
- Check ~/ in Powershell
6.2.0> Resolve-Path ~/ Path ---- C:\Users\xiang
So if we created the .pypirc file in
~/ in Powershell, twine won’t find it.
Why os.path.expanduser() doesn’t resolve the same ~/ as Powershell
As shown previsouly, Windows Powershell resolves
$env:USERPROFILE. How about os.path.expanduser()? Let’s check its source code by the
In : import os ; print(inspect.getsource(os.path.expanduser)) def expanduser(path): """Expand ~ and ~user constructs. If user or $HOME is unknown, do nothing.""" path = os.fspath(path) if isinstance(path, bytes): tilde = b'~' else: tilde = '~' if not path.startswith(tilde): return path i, n = 1, len(path) while i < n and path[i] not in _get_bothseps(path): i += 1 if 'HOME' in os.environ: userhome = os.environ['HOME'] elif 'USERPROFILE' in os.environ: userhome = os.environ['USERPROFILE'] elif not 'HOMEPATH' in os.environ: return path else: try: drive = os.environ['HOMEDRIVE'] except KeyError: drive = '' userhome = join(drive, os.environ['HOMEPATH']) if isinstance(path, bytes): userhome = os.fsencode(userhome) if i != 1: #~user userhome = join(dirname(userhome), path[1:i]) return userhome + path[i:] In :
From the source code, obviously, if
$env:HOME exists, expanduser() will return its value. If
$env:HOME doesn’t exists, it falls back to
$env:USERPROFILE, if not again, it falls back to
We have 3 solutions.
twine --config-fileto manually specify the .pypirc config file.
if $env:HOME exists, copy the .pypirc file to $env:HOME, otherwise $env:USERPROFILE.
declare all the upload params as environment variables.
Leave a comment