The scripts are also provided as part of a tarball. but agreed that isnt very nice.

Most of our script users still use git to fetch/sync library and scripts together, otherwise their scripts fail.

It would be nice to have a console script pwb.py which can find the script in different packages. I've commented more on T139143: Make a pywikibot entry point for scripts, but one approach is to create a package namespace , and that package namespace can provide the pwb.py.

I think @valhallasw created a task for moving scripts out of the -core repo, but I cant find it now.

So ideally we have a mechanism in place that allows scripts to be moved from an omnibus package into a separate package, and the user invocation is identical.

We did create a pywikibot-wiktionary to avoid that chunk of code landing in -core, but as I recall it has very little pywikibot usage. It should probably be split into a wiktionary library and a script that utilises both the wiktionary and pywikibot libraries.

Ideally we release 2.0 final, and then request that people put their hands up to be maintainers of a single script. There are many scripts that have one primary author over the years - that person may be happy to take more ownership, especially if we can set up the repo /packaging for them to maintain.

And then the remainder of the scripts could be grouped into sets of scripts with a common focus, e.g. Pywikibot-Wikidata , Pywikibot-Commons , etc.

See also T115784

Change 842959 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] Provide an entry point to connect foreign scripts with pwb wapper

https://gerrit.wikimedia.org/r/842959

I don't particularly care how the various bits of the distro is broken into pieces, but for anyone who primarily wants to use the finished scripts the PyPI package is a bit useless right now. Installing the pywikibot package and then having to do a git checkout of it too in order to use it is… well, let's call it "not optimal" and leave it at that.

In terms of use cases, the number of people who can and want to use the officially supported scripts > the number of people who primarily want to write their own scripts (use pywikibot as a pure framework) > the number of people who want to develop pywikibot itself. The git checkout (or tarball download, or…) should only be necessary for the latter group; and if you can only optimize for one of the former then pick the first group. You can still use pywikibot as a pure framework even if the scripts are included in the package, but you can't (sensibly) use the scripts if they aren't available at all.

In other words, the precise packaging is really an optimization problem where you need to find a good balance between different needs. First step is to make the scripts available in the package at all, then consider whether they should be split out into separate packages to avoid annoying the purists^W^W^W make dependency management easier and reduce installed footprint. But the scripts do need to be available in a package (ASAP) irrespective of whether that package is pywikibot, pywikibot-scripts, or pywikibot-adminscripts.

I ran into this yesterday because I needed to sysop-protect a bunch of pages and my usual go-to PAWS apparently lacks the right OAuth grants. So I switched to the local git checkout on my mac, git pull origin master, and try to run pwb.py protect it in place only to have it throw due to lacking the requests library. So I turn to Homebrew (a macos package manager) to install it without cluttering up the system python, only to find that because of reasons they have stopped distributing Python libraries and are suggesting either pip inside a venv or pipx (that handles the venv for you). So…

> pipx install pywikibot
> pipx inject pywikibot requests

…and then figure out that the scripts are for some inexplicable reason not included. I happen to have a git checkout so I can run the scripts there, but had that been the next step in the installation instructions I would have just dropped it. Having to also pipx inject pywikibot pywikibot-scripts is reasonable, but git is not a tool you should point non-developers at.

PS. Either way the docs need to be updated. The statement "Scripts are only available with Pywikibot if installed in directory mode and not as site package"—which, as best I can tell, is trying to communicate that the scripts are not included in the PyPI package—is impenetrable. Neither "directory mode" nor "site package" or universally understood terms and are not explained or defined in the docs.

@Xover: thank you for this valuable comment. Distributing the script package is not trivial and was not able to implement in a good way before Python 3.7. I am still working on this task from time to time but there are other issues which might be more important. Anyway there is a way to get the whole package of Pywkibot together with their scripts via the nighly dump at https://pywikibot.org. Does this help in mean time?

Neither "directory mode" nor "site package" or universally understood terms and are not explained or defined in the docs.

Good point! Looks like developer slang who knows their meaning but in others should not guess what it means.

In T139144#10067083, @Xqt wrote:

… there are other issues which might be more important.

To put it somewhat more dramatically than is otherwise warranted for rhetorical purposes: this issue is preventing a group of users from using that functionality of pywikibot when distributed it as a PyPI package. I imagine the list of stuff that needs doing is near infinite, so the need to prioritise is obvious, but count this as an argument that this issue does have a certain level of urgency to it too.

Anyway there is a way to get the whole package of Pywkibot together with their scripts via the nighly dump at https://pywikibot.org. Does this help in mean time?

In the sense that a zip download is somewhat less likely to scare people away than pointing them at git, sure. But as mentioned, what sent off down this rabbit hole was the fact that pywikibot has a dep on the requests library so the tarball isn't standalone. At which point you're messing with multiple ways to install stuff and manage deps to get pywikibot scripts running. I had an existing git checkout sitting around since T300531 that I could update and just point pwb there, but this all was a lot more hassle than it should have been and required me to ask stupid questions on IRC to figure out what was going on.

\o/

Here is a prerelease package for pywikibot-scripts and pywikibot. Both have the version 9.4.0. I propose to sync the pywikibot-scripts version vith its framework. Are there other suggestions? pywikibot-scripts is only available as wheel distribution, not tarball. I didn't found a way to create it yet.

❯ pipx install pywikibot-9.4.0.dev1-py3-none-any.whl
  installed package pywikibot 9.4.0.dev1, installed using Python 3.12.5
  These apps are now globally available
    - pwb
done! ✨ 🌟 ✨
❯ pipx inject pywikibot requests_oauthlib                            
  injected package requests-oauthlib into venv pywikibot
done! ✨ 🌟 ✨
❯ pipx inject pywikibot_scripts-9.4.0-py3-none-any.whl 
Error: 'pywikibot_scripts-9.4.0-py3-none-any.whl' looks like a path. Expected the name of an installed package.
❯ pipx install pywikibot_scripts-9.4.0-py3-none-any.whl

No apps associated with package pywikibot-scripts or its dependencies. If you are attempting to install a library, pipx should not be used. Consider using pip or a similar tool instead.
❯

Not sure how that can be addressed. Perhaps either pywikibot-scripts needs to be the package that exposes the pwb command (and then has a hard requirement on pywikibot so it always gets installed as a dep when pywikibot-scripts is), or possibly that each script contained in the package should be exposed as an "app" to pipx?

I am not familiar with pipx. Currently pywikibot-scripts does not have any requirements for tests but pywikibot will be added to requirement list later with the final release. The pwb.exe is a console entry point which calls a wrapper script to invoke any script in predefined script folders, and it is already part of the pywikibot framework. pywikibot-scripts installs a module entry point like in flake8 or other packages, so that the wrapper can use it as a scripts folder path. Therefore the pwb.exe couldn't be part of the pywikibot-scripts; other script libraries are also conceivable to use the entry-point mechanism. Installing pywikibot` and pywikibot-scripts locally via pip install ... works for me as expected. Maybe you could use venv and take pip as installer.

Change #842959 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Provide an entry point to connect foreign scripts with pwb wapper

https://gerrit.wikimedia.org/r/842959