Wikipedia:Bots/Requests for approval/DeadLinkBOT: Difference between revisions

{{Newbot|DeadLinkBOT}}

 

 

 

<!-- This is not a vote. It is a discussion -->

:What about websites that go through regular downtime? If the bot reads them as dead while they are temporarily down, it will remove a good link. [[User:X!|<span style="font-family:Verdana,Arial,Helvetica;color:steelblue;">'''X'''</span>]][[User talk:X!|<span style="color:steelblue;"><small>clamation point</small></span>]] 05:01, 2 December 2008 (UTC)

:::::Yes, I will add that feature. --[[User:ThaddeusB|ThaddeusB]] ([[User talk:ThaddeusB|talk]]) 22:00, 7 December 2008 (UTC)

 

::I wasn't planning on releasing it for public consumption. --[[User:ThaddeusB|ThaddeusB]] ([[User talk:ThaddeusB|talk]]) 22:00, 7 December 2008 (UTC)

::::Well considering its explicitly not required, I shouldn't have to justify my decision. But since you asked, my code is undocumented and "ugly" - it is not intended to be read by anyone but me. I really don't see what the issue is - all the program does as far as Wikipedia goes is substitute a pre-screened dead URL for a pre-screened good one, possibly applying pre-screened regrexes to pick between two or more different options. None-the-less, I put the code up anyway: [[User:DeadLinkBOT/source]] --[[User:ThaddeusB|ThaddeusB]] ([[User talk:ThaddeusB|talk]]) 23:45, 12 December 2008 (UTC)

 

::I also stated the list of links I'd be starting with above. This is from a specific request from [[Wikipedia:AutoWikiBrowser/Tasks#LeighRayment.com_.28continued.29]]. There are 2500+ of them. I have tested the first batch of them with local writes, but it's against WP policy to have a bot edit WP without test approval, so of course I haven't actually written them to WP. Isn't that the whole point of having a test period?

::Since its correcting DEAD links, I don't see any reason to limit its scope (although it does avoid editing archives), but I could easily change that if desired. --[[User:ThaddeusB|ThaddeusB]] ([[User talk:ThaddeusB|talk]]) 05:13, 15 December 2008 (UTC)

::::I wrote the bot in order to correct the angeltown links, but I don't see any reason to limit its scope. I have written dozens of text-parsing scripts in the past and am well aware of the potential issues involved with unexpected input and such. I do realize AWB-style general fixed are difficult to implement correctly, but I feel I am up to the challenge. Nonetheless, I will drop that part of the request at this time. (I have seen bot approved for gen fixes in the past and didn't htink it would be an issue or I would never have added that part.) HTML links, however, do not present such issues. There just is not any realistic chance of a search for "http://www.somesite.com/directory/somedeadURL.htm" generating false positives outside of a few specific pages such as WP's list of dead links. If my bot works correctly on somewebsite.com, it will work on someotherwebsite.com assuming the input (supplied by me) is valid.

::::I am well aware that ultimately I am responsible for every edit the bot makes, and will utilize the utmost care in what I tell it to fix. If a maliciously tell it t change every www.microsoft.com to www.myspamsite.com then obviously I'd be in trouble. But if that was my intention why would I even bother trying to get approval? --[[User:ThaddeusB|ThaddeusB]] ([[User talk:ThaddeusB|talk]]) 12:37, 15 December 2008 (UTC)

Can a member of BAG please explain exactly what they want me to do to prove this bot works correctly? I've tested it locally, answered every question here, released the source, and tried to be patient but no one seems to be willing to act. What do I need to do to get the ball rolling? --[[User:ThaddeusB|ThaddeusB]] ([[User talk:ThaddeusB|talk]]) 02:34, 18 December 2008 (UTC)

 

*It can be used for "identification" situations to quickly identify all occurrences of a particular URL or URL fragment for other uses, such as looking for patterns of spamming, etc. I'm not familiar with semi-automated editing tools, but in principle the generated list can be used as input to a semi-automated tool, leaving it to a human being to confirm or cancel each edit. This would be practical on only relatively short lists, maybe a few hundred or so.

*The read-only portion by definition does not need approval of the BAG, it can run as soon as it's written.

 

 

:Hello, the way the bot is currently structured is as follows:

<blockquote>

# Write recommended changes to file for review

'''Wait a week''' to insure the URL is indeed dead

# Get URL + change(s) from file; change can require a simply test such as making sure "text" is in the page to be changed and make decisions based on those tests; the new text can be anything - presumably a URL or a template.

# Use [[Special:LinkSearch]] to find all instances of the URL on wikipedia - this list could be output to a file if you want

::*After a suitable number of changes are in the holding file, manually review each change and mark it approved. A "suitable number" could be 1 change or an entire batch. Doing it 1 change at a time simulates assisted-editing tools like AutoWikiBrowser.

::*For each approved change, verify there have been no intermediate edits and make the edit then move on to the next approved change. If possible, don't count intermediate edits that only affected other sections, i.e. make the change if at all possible, but abandon any change that looks like an edit-conflict and log it as a failure so it can be done over again.

:::Thanks for the comments.

:::*I can certainly review the first X changes manually before uploading. However, the bot is capable of doing, for example, 1000 edits in under 3 hours (with standard rate limits applied); I certainly don't want to review 1000+ edits, let alone an entire month's worth. (I have already written and reviewed a # locally, but can also intermediately review some more.)

:::*I think you were actually talking about only having the edit conflict feature for the manually approval tests which is definitely wise. However, once it goes live, it would be pointless. I could pull the history and check for intermediate updates, but this would most likely actually take longer than the text parsing (which happens in a tiny fraction of a second.) I could, however, pull the history after an edit to just make sure there was no intermediate edit and auto-revert if there was any. LMK what you think. --[[User:ThaddeusB|ThaddeusB]] ([[User talk:ThaddeusB|talk]]) 21:26, 18 December 2008 (UTC)