packages/lang/python/python-package-install.sh, branch master

python: Rework filespec install script

2023-08-24T17:47:43Z

* Support wildcards in install (`+`) paths * Add fourth parameter to set directory permissions If file permissions are given (third parameter), these will now apply to files only. * Add non-recursive set permissions command (`==`) * Be more strict about filespec format Blank lines and lines starting with `#` will be ignored. Other errors (unknown command, missing path parameter, etc.) will cause the script to exit. * Be more strict about ensuring paths exist for all commands * Avoid spawning subshells This also removes outdated filespec paths in the python3 package; these paths delete files that are no longer present. Signed-off-by: Jeffery To

python3: Move functionality into python3-package.mk

2020-04-18T17:56:23Z

This moves functionality from python-package-install.sh into python3-package.mk, so that they can be reused separate from filespec processing. Signed-off-by: Jeffery To

python,python3: add Py[3]Shebang functions & move outside of script

2019-09-20T11:10:00Z

Some packages just install some Python binaries, that may need their shebang fixed. This change adds some utilities to help with that and try to centralize the sed rules a bit. It also removes the logic from the `python-package-install.sh` into the `python-package[3].mk` files. This does 2 things: 1. It minimizes the need for the shell script to know the Python version 2/3 2. Makes the logic re-usable in packages; especially if the install rules differ a bit Signed-off-by: Alexandru Ardelean

python: change condition check for legacy opt for Python3

2019-09-20T10:42:21Z

As I remember this worked. But since `set -e` is set, I am a bit paranoid about it. In the sense that it may fail if `ver` != 3. Signed-off-by: Alexandru Ardelean

python,python3: Increase max recursion level when generating bytecode

2019-04-03T19:07:02Z

"python -m compileall" has a default maximum recursion level of 10, i.e. it will descend up to 10 levels of subdirectories when looking for source files to compile. This is usually sufficient but there are packages that include more than 10 levels (botocore, https://github.com/openwrt/packages/pull/8214#discussion_r270056741). This adds the "-r" command line option to the call to compileall to increase the max recursion level (currently set to 20). This also patches Python 2's compileall.py to add this max recursion level option. (Python 3's compileall.py already supports this option.) This also applies some related changes to python-package-install.sh: * Use the "-delete" option with find instead of exec'ing rm / rmdir. For the case of removing empty directories (in delete_empty_dirs()), this has the added benefit of simplifying the code, as the "-delete" option implies "-depth", and thus find "does the right thing" (removing empty directories depth-first). * Remove the backslash in "-name" patterns (for find), as they are not regular expression but glob patterns. Signed-off-by: Jeffery To

python,python3: Fix overridden usr/bin symlinks

2019-04-03T13:05:14Z

Currently, all files in usr/bin (presumably all Python scripts) are run through sed to replace the shebang; sed will overwrite the file whether or not a match is found. This causes symlinks to be overridden and made into copies of their targets. python[3]-base and python[3]-dev are affected by this. This adds the --follow-symlinks flag to sed, in addition to using $(SED), so that symlinks are not overridden. Signed-off-by: Jeffery To

python,python3: move shebang handle in install script

2019-02-12T10:31:55Z

This extends the Python[3] shebang fixup to all packages. Only Python scripts in `/usr/bin` will be handled at the moment. Later it may make sense to also cover executables in `/bin`, though typically Python executables shouldn't be placed there. Previously the shebang handling was only done for python[3]-pip & python[3]-setuptools. Signed-off-by: Alexandru Ardelean

python,python3: make deletion tolerant for paths with spaces

2019-02-12T10:31:54Z

Piping to xargs does not handle spaces in paths too well, because it splits up the paths. For deleting empty dirs, we also need to do several retries, otherwise `find` will try to go through the directories after they're deleted. Signed-off-by: Alexandru Ardelean

python,python3: move .exe removal in `python-package-install.sh` script

2019-02-04T14:38:12Z

It's a common operation for both Python & Python3, so move it to the script `python-package-install.sh` script. Signed-off-by: Alexandru Ardelean

lang/python/python-package-install.sh: assign SOURCE_DATE_EPOCH to PYTHONHASHSEED

2018-03-07T18:52:15Z

Following a discussion on bugs.python.org: * https://bugs.python.org/issue29708 * https://bugs.python.org/msg313384 It seems that setting a fixed value to PYTHONHASHSEED guarantees that the bytecodes are generated consistently/in a reproducible manner. Hopefully, this is the last bit to make Python3 build reproducible. Tested this locally on a few files [that were not reproducible without this change]. The PYTHONHASHSEED is only assigned to the host Python/Python3 during compilation of byte-codes [from python source]. Signed-off-by: Alexandru Ardelean