Wikipedia:Bots/Noticeboard

Overview Dictionary Bot policy Bot Approvals Group Guide Noticeboard Newsletter Bot Requests Requests for Approval Adminbots {{BAG Tools}} Creating a Bot List of bots Activity monitor Status report History Types

Hello. I am not sure if this is the correct venue. If this cant be solved here, kindly let me know where should I go.
I currently have the AWB bot on enwiki, User:KiranBOT. It adds wikiproject banners on talkpage (simplest task, I think). In short: I need to create a fully automated/toolforge bot.

Prelude:Around 20 days ago, I got bot flag on mrwiki (AWB). Within less than 20 days (in around 7-8 runs), it racked-up more than 10k edits there (mr:special:contributions/KiranBOT). Because of the syntax of Marathi language, and word rules (not grammar rules), there are many uncontroversial find and replace tasks. But there are less than 10 active/regular editors, so such tasks have been piled up.

To the point: On mrwiki, I would like to run a simple bot — but the one with continuous editing, like DumbBOT. Few hours ago, I created an account on wikitech/toolforge, and requested for membership. But I am still not sure how, and where to upload the bot's code. I want to code it in C#. The bot will obviously be discussed/vetted on mrwiki, along with the keywords to be replaced (I have created a rudimentary list at mr:User:Usernamekiran/typos). Any help/guidence will be appreciated a lot. —usernamekiran • sign the guestbook • (talk) 23:38, 31 December 2021 (UTC)[reply]

Hey there. Here's my notes for PHP programming language: User:Novem Linguae/Essays/Toolforge bot tutorial. In particular, you can use this to get your FTP client and console running. For C#, I think you would want to use this: wikitech:Help:Toolforge/Mono. And also Grid instead of Kubernetes. Hope that helps. –Novem Linguae (talk) 23:56, 31 December 2021 (UTC)[reply]

thanks. looking into it. —usernamekiran • sign the guestbook • (talk) 23:59, 31 December 2021 (UTC)[reply]

The Novem Linguae tutorial looks good to get started, but two things to note:
1. It mentions use of an FTP client to transfer files to the host. There's another way – git. You can set up a repository on GitHub/Gitlab and push your code there. On the toolforge host, you can pull it. There are ways, using webhooks or GitHub Actions, through which you can even trigger pulls automatically on TF when you push locally.
2. It mentions use of Kubernetes for cron jobs. Using the grid is much easier (requires just adding a line to the crontab file). – SD0001 (talk) 04:01, 1 January 2022 (UTC)[reply]

dummy comment to avoid archiving. —usernamekiran • sign the guestbook • (talk) 18:41, 17 January 2022 (UTC)[reply]

So I could transfer files using github, and also created files using mono on putty/CLI. But I couldnt execute the bot. First I went with C#, then python, but both didnt work. I have lots of material in .net to study/refer like dotnetwikibot framework, source code of AWB, and some other programs mentioned at mediawiki. All I need is a little guidance regarding how to compile and run it on toolforge. Your help will be appreciated a lot. Also pinging @Mz7, JPxG, and ST47: —usernamekiran • sign the guestbook • (talk) 15:44, 18 January 2022 (UTC)[reply]

Page watchers may be interested in

Would a BAGger please look at Wikipedia talk:Bots/Requests for approval/Fluxbot 6 and advise? Thank you! — xaosflux ^Talk 16:59, 19 January 2022 (UTC)[reply]

Lots of categories named "NPOV disputes from Month Year" have been moved to the "Wikipedia neutral point of view disputes from Month Year" name by

WP:CFDW since manual work is required. (See bolded note at the top of the bot work section.) I've reverted the addition which will stop my bot. — JJMC89 (T·C) 21:31, 19 January 2022 (UTC)[reply

]

I have requested undeletion at

talk) 22:22, 19 January 2022 (UTC)[reply

]

As I appear to be the one that has caused the problem I will sort it out, though I am unsure what needs to be done. After I manually rename the cats, what should I then do to prevent problems?

GeoffreyT2000, JJMC89, are you able to advise? SilkTork (talk) 08:47, 20 January 2022 (UTC)[reply

]

As the categories are populated by templates, what you'll need to do is update the relevant templates to populate the new categories. You'll also need to move Category:NPOV disputes itself and the related stuff referred to at Wikipedia:Creating a dated maintenance category so AnomieBOT will work correctly.

There's probably no need to worry about the "NPOV disputes from Month Year" categories themselves, once you've done the other work they'll be emptied and someone (e.g. Fastily) will delete them, while AnomieBOT will create the new "Wikipedia neutral point of view disputes from Month Year" categories as they're populated. Just check later on that they did eventually get emptied and deleted. Anomie⚔ 13:20, 20 January 2022 (UTC)[reply]

I clearly should not have stepped into this. It seemed a simple closure, and it turns out to be somewhat more complicated and esoteric. Would someone advise me what the "relevant templates" are and how to update them "to populate the new categories". SilkTork (talk) 18:40, 20 January 2022 (UTC)[reply]

Dated maintenance categories aren't the most straightforward to deal with. I think I got everything that needed doing. — JJMC89 (T·C) 02:59, 21 January 2022 (UTC)[reply]

This is why I haven't gone into closing CFD discussions, they can have repercussions. Liz ^{Read! Talk!} 04:17, 21 January 2022 (UTC)[reply]

Thanks JJMC89. Yes, I can now see why that discussion was unclosed after two months even with a clear consensus to rename. My attention was drawn to the discussion by a pink link, and after reading it I was going to add my support for a rename, but when I saw how old the discussion was, I thought the more appropriate thing to do was close it. And then I followed the instructions for what to do after closing such a discussion. I probably missed some instructions on that page for special cases like this. Anyway, lesson learned - like Liz I shall stay away from closing CFD discussions in future! SilkTork (talk) 08:57, 21 January 2022 (UTC)[reply]

This is not what should be happening. The overwhelming majority of CFD discussions can be closed by just listing the categories at CFDW, or at least doing so will not cause a disaster. * Pppery * _{it has begun...} 15:12, 21 January 2022 (UTC)[reply]

Well, * Pppery *, I think there needs to be a "Dummies Guide" to closing CfD discussions because I've read over the instructions several times over the past few years and found them intimidating enough that I never started doing closings. I think I need someone to walk me through it and to admit that is humbling. Liz ^{Read! Talk!} 02:40, 25 January 2022 (UTC)[reply]

It's really not that hard.

• For discussions where the result is "Keep" or "No consensus" , you can just use XfDCloser, which should handle all required actions.
• For discussions where the result is "Delete", you can use XfDCloser to close the discussion, and then list the name of the category at Wikipedia:Categories for discussion/Working#Empty then delete in the format specified in HTML comments on that page. For some categories additional manual actions are necessary, but nothing will go wrong if the category sits there and you let one of the people who watch that page do the manual work.
• "Merge" results follow the same procedure (with the same caveat), except the section to use is Wikipedia:Categories for discussion/Working#Merge then delete
• "Rename" results can be done following a similar procedure (using Wikipedia:Categories for discussion/Working#Move). There's slightly more potential for things to go wrong there, because the bot moves the category without leaving a redirect and tries to update all uses. If it fails for whatever reason, then the result will be pages in red-linked categories. Usually they get fixed properly by whoever patrols that list (or a CFDW watcher), but occasionally something goes wrong.

I can say for sure that it is not possible to recreate this specific mess (a "rename" case in which an incomplete implementation of the rename caused clashes with other bots) by closing any of the pending CfD discussions. * Pppery * _{it has begun...} 03:00, 25 January 2022 (UTC)[reply]

Why do the instructions not say this simple version? I had the same reaction as Liz. :) Izno (talk) 03:45, 25 January 2022 (UTC)[reply]

I'm not the person who wrote the instructions, so don't know for sure, although my hunch would be they predate XfDCloser, and also the way I phrased it delegates some of the hard work to other people. Feel free to copy some version of what I wrote above to that page if you want to. * Pppery * _{it has begun...} 03:53, 25 January 2022 (UTC)[reply]

* Pppery *, I see an exciting career ahead of you in software documentation! Wooo! I don't know why the CFD instructions are unnecessarily confusing and complex, you make it sound very straightforward. Many thanks. It's on my "To Do" list to try to take on some more simple closures next month. Liz ^{Read! Talk!} 05:17, 29 January 2022 (UTC)[reply]

I guess that would be down to me in most part, to address various things that went wrong in the past. Nice work, Pppery! I will look at further updating the instructions after I've had some practice using XFDCloser. – Fayenatic London 17:09, 2 February 2022 (UTC)[reply]

• I've blocked the bot from editing Wikipedia:Miscellany for deletion/Archived debates/January 2022 for a month, pending resolution. Any admin (including @Legoktm:) is welcome to unblock if resolved. — xaosflux ^Talk 22:27, 26 January 2022 (UTC)[reply]
  • Operator notified on talk. — xaosflux ^Talk 22:28, 26 January 2022 (UTC)[reply]

Bleh, thanks for the report and partial block, looking into it now. Legoktm (talk) 04:54, 27 January 2022 (UTC)[reply]

• @
  (on the go!) 14:57, 3 February 2022 (UTC)[reply
  ]

@

PorkchopGMX (on the go):  Done — xaosflux ^Talk 15:43, 3 February 2022 (UTC)[reply

]

I disabled the automatic running of the MFD archiver script. I have some new code which should be ready to go, but I'll run it manually for a few days before automating it. There are no MFDs needing archiving right now so I'll give it a shot tomorrow and lift the partial blocks then. Legoktm (talk) 07:19, 4 February 2022 (UTC)[reply]

Scottywong, when should obsolete HTML tags be converted to modern syntax? Lint errors have been flagged by MediaWiki since 2018, so a small group of editors have already been fixing errors for over three years and there are still millions of errors. Given that we have fixed a lot of the easy errors, the remaining millions of errors will take multiple years to fix. – Jonesey95 (talk) 16:13, 28 January 2022 (UTC)[reply]

It is properly tagging the edit as "bot" and "minor" - so watchlist flooding should be able to be alleviated by hiding bot edits. — xaosflux ^Talk 16:23, 28 January 2022 (UTC)[reply]

I understand why the bot is making these edits, and how to hide them from my watchlist. However, if you're suggesting that

WP:BOTPOL? Or can you explain how this bot is not making purely cosmetic edits to the wikitext of pages? —⁠ScottyWong⁠— 16:40, 28 January 2022 (UTC)[reply

]

I haven't gone through that part, was looking if there was any immediate tech issue that was causing flooding. — xaosflux ^Talk 16:47, 28 January 2022 (UTC)[reply]

WP:COSMETICBOT explicitly mentions [fixing] egregiously invalid HTML such as unclosed tags, even if it does not affect browsers' display or is fixed before output by RemexHtml (e.g. changing <sup>...</sub> to ^...) as non-cosmetic. – SD0001 (talk) 16:47, 28 January 2022 (UTC)[reply

]

WP:COSMETICBOT. – Jonesey95 (talk) 16:49, 28 January 2022 (UTC)[reply

]

The BRFA was speedily approved in less than 3 hours, with no opportunity for community discussion. This discussion can act as a test for whether or not there is community consensus for this bot to operate in violation of

WP:COSMETICBOT. The <sub></sup> example given above is substantive, because it would actually change the way the page is rendered. Changing <font> tags to <span> tags results in no change whatsoever, since every modern browser still understands and supports the <font> tag, despite it being deprecated. —⁠ScottyWong⁠— 16:54, 28 January 2022 (UTC)[reply

]

Well, unlike the hard work we did to clear out obsolete tags in Template space, we're not going to fix millions of font tags in talk space pages by hand, which leaves two options that I can see: an automated process, or leaving the font tags in place until it is confirmed that they will stop working. It sounds like what you want is an RFC at VPR or somewhere to ask if we should formally deprecate the use of font tags on the English Wikipedia. You might want to ask about

other obsolete tags (<tt>...</tt>, <strike>...</strike>, and <center>...</center>) while you're at it. – Jonesey95 (talk) 17:02, 28 January 2022 (UTC)[reply

]

There's a difference between "from this point on, let's not use font tags anymore" and "let's go back to millions of dormant AfD pages (most of which will never be read or edited ever again, for the rest of eternity) and make millions of edits to change all of the font tags to span tags." Let's see how this discussion goes first, and then we can determine if a wider RFC is necessary. —⁠ScottyWong⁠— 17:15, 28 January 2022 (UTC)[reply]

My bot edits are not in violation of

WP:COSMETICBOT, Lint errors are exempt from the usual prohibition on cosmetic edits. See point 4 fixed before output by RemexHtml covers Lint errors.

As for the speedy approval, the context for that is the prior BRFAs for MalnadachBot. They were to fix very specific types of Lint errors that were all done successfully after testing and discussion, fixing over 4.7 million Lint errors in the process. ಮಲ್ನಾಡಾಚ್ ಕೊಂಕ್ಣೊ (talk) 17:18, 28 January 2022 (UTC)[reply

]

Perhaps this is a pedantic point that misses the crux of what you're saying, but there are millions of errors, not millions of AfDs (per a Quarry there are only 484,194 AfD subpages, excluding daily logs). jp×g 20:39, 31 January 2022 (UTC)[reply]

Regarding the speedy approval: the bot operator had 10 successful similar runs fixing these types of errors, so to say that there was "no opportunity for discussion" is a little silly - the first task was approved in May 2021, so in my mind that is 9 months worth of bot edits during which the task(s) could have been discussed. When a bot operator gets to a point where they have a bunch of similar tasks that are trialled and run with zero issues, I start speedy-approving them, not only because it saves the botop time, but it has been demonstrated that the type of tasks being performed by the bot are not an issue. Primefac (talk) 17:19, 28 January 2022 (UTC)[reply]

To be clear, I'm not saying that the speedy approval was necessarily inappropriate. I was responding to Jonesey95's assertion that the BRFA represents explicit community consensus for this task. I'm also not suggesting that the bot is doing anything technically incorrect, or that it has any bugs. All I'm suggesting is that if fixing purely cosmetic wikitext syntax issues on ancient AfD pages doesn't qualify as

WP:COSMETICBOT no longer reflects the way that bots work on WP, then perhaps we should remove it. But until it's removed, I still believe that this type of task falls on the wrong side of bot policy, as currently written. —⁠ScottyWong⁠— 17:32, 28 January 2022 (UTC)[reply

]

I quoted the relevant portion of COSMETICBOT above. – Jonesey95 (talk) 17:48, 28 January 2022 (UTC)[reply]

Speaking only for myself, I have a hard time finding a basis revoking approval (or denying it). The bot is doing the legwork to future proof our pages with deprecated syntax. That to me, is a good thing. The bot op / bot page however, could mention

b} 17:50, 28 January 2022 (UTC)[reply

]

I agree with Primefac's assessment here. A dozen BRFAs that are preventative measures to avoid pages falling into the state where we would find COSMETICBOT irrelevant, never mind the lines in COSMETICBOT that indicate that these changes can be executed today? Sounds reasonable to me. It also avoids (good faith) time spent elsewhere like at

WP:VPT when we get questions about why someone can't read an archive. Izno (talk) 18:51, 28 January 2022 (UTC)[reply

]

It seems that I'm the lone voice on this one (apart from one other user that expressed concern on the bot owner's talk page), which is fine. If you wouldn't mind, I'd like to leave this discussion open for a while longer to give anyone else an opportunity to express an opinion. If, after a reasonable amount of time, there is clear consensus that making these cosmetic changes is a good thing for the project, I'm happy to respect that and crawl back into my hole. I paused the bot while this discussion was ongoing; I will unpause the bot now since it seems somewhat unlikely that approval for this task will be revoked, and allowing the bot to continue making these edits might draw more attention to this discussion. —⁠ScottyWong⁠— 18:56, 28 January 2022 (UTC)[reply]

Just a quick note from my phone, re: COSMETICBOT: Something that would qualify as a cosmetic edit that would probably never gain approval would for example be aligning '=' signs in template calls (as you often see in infoboxes) or or removing whitespace from the ends of lines. These things might clean up the wikitext, but they don't change the HTML output. AFAIK, that's what COSMETICBOT is good for. --rchard2scout (talk) 15:09, 30 January 2022 (UTC)[reply]

I agree, cosmetic bot should be used with common sense it's not a hard red line. The question is this bot a good idea, enough to justify so many edits, and the answer is yes IMO. Furthermore, while the changes may technically be cosmetic today they won't be in the future, presumably, if/when some browsers stop supporting older syntax. I wish we had a way to make these types of edits hidden by default vs. opt-in. -- GreenC 15:24, 30 January 2022 (UTC)[reply]

... make these types of edits hidden by default... - they already are, as the default email watchlist settings are to hide minor edits. Hell, if it's a minor bot edit, it will keep it off your watchlist even if you do want it to show up. Primefac (talk) 20:48, 30 January 2022 (UTC)[reply]

Are you sure? At the top of my watchlist I have an array of checkboxes to hide:

I see all minor and bot edits to articles in my watchlist. Were it true that minor and bot edits are default hidden,

Monkbot/task 18

might have run to completion.

—Trappist the monk (talk) 21:04, 30 January 2022 (UTC)[reply]

Apologies, should have specified I was referring to email notifications. I have updated my comment accordingly. Primefac (talk) 21:09, 30 January 2022 (UTC)[reply]

I understand where everyone is coming from, and I don't intend to continue arguing about it when it's clear I'm in the minority, but perhaps I'm a little out of the loop. Here's my question: is there any real evidence that any major, modern browsers have plans to fully deprecate support for things like <font> tags and other HTML4 elements? Will there come a time that if a browser sees a font tag in the html source of a page, it literally will ignore that tag or won't know what to do with it? It seems like such an easy thing for a browser to continue supporting indefinitely with little to no impact on anything. I suppose I'm wondering if this bot is solving an actual problem, or if it's trying to solve a hypothetical problem that we think might exist at some point in the distant future. —⁠ScottyWong⁠— 21:28, 30 January 2022 (UTC)[reply]

That is a great question to address to MediaWiki's developers, who have deliberately marked <font>...</font> and a handful of other tags as "obsolete" by inserting error counts into the "Page information" page for all pages containing those tags. In the software world, marking a specific usage as obsolete or deprecated is typically the first step toward removal of support, and the MW developers have removed support for many long-supported features over the years. The MediaWiki developers may have similar plans for obsolete tags, or they may have other plans. – Jonesey95 (talk) 23:30, 30 January 2022 (UTC)[reply]

There are some good notes at Parsing/Notes/HTML5 Compliance. There's also some discussion buried in gerrit:334990 which mostly represents my current thoughts (though I certainly don't speak for the Parsing Team anymore), which is that it is probably unlikely browsers will stop supporting <font>, <big>, etc. in the near future. If they do, we could implement those tags ourselves to prevent breakage either at the wikitext->HTML layer or just in CSS. I don't think there are any plans or even considerations to remove these deprecated tags from wikitext before browsers start dropping them. I said this in basically the same discussion on Meta-Wiki in 2020.

That said, I would much rather see bots that can properly parse font/etc. tags into their correct span/etc. replacements so it can all be done in one pass instead of creating regexes for every signature. Legoktm (talk) 18:49, 4 February 2022 (UTC)[reply]

This is not a hypothetical problem, we know for sure that at least one obsolete html tags marked by Linter does not work in mobile. <tt>...</tt> renders as plain text in Mobile Wikipedia. Compare this in mobile and desktop view. Based on this we can reasonably conclude that <font>...</font> will stop working at some point as well. Besides not everything that is obsolete in HTML5 is counted as obsolete by Linter. For example tags like <big>...</big> and table attributes like align, valign and bgcolor are not marked by Linter even though they too are obsolete in HTML5 like font tags. So it seems the developers have plans to continue support for these, but not for font tags. ಮಲ್ನಾಡಾಚ್ ಕೊಂಕ್ಣೊ (talk) 06:00, 31 January 2022 (UTC)[reply]

Although for the record, <tt>...</tt> does not work in mobile because of specific css overriding it in the mobile skin (dating back to 2012). Whether any common mobile browsers didn't or don't support it is unclear. Anomie⚔ 13:57, 31 January 2022 (UTC)[reply]

So, it still sounds like we're fixing hypothetical problems that we think will eventually become real problems. I agree that it'll probably eventually become a real problem one way or the other, but honestly I still don't see the point of correcting problems in someone's signature on a 12 year-old AfD. —⁠ScottyWong⁠— 07:17, 1 February 2022 (UTC)[reply]

As much I don't like my watchlist full of bot edits, it's not a COSMETICBOT violation as long the changes clear a maintenance error. What I would say is a problem is that the bot doesn't actually fix all the issues at once. For example, this edit is fine, but what about these font tags? It the bot really going to come back and edit the page again? Or even the same task like this edit, which is fine, except there are two other font tag uses. Is the bot really replacing each signature one at a time? —  HELLKNOWZ ∣ TALK 11:12, 31 January 2022 (UTC)[reply]

This is, to be honest, one of the reasons why I gave a slightly-more-blanket approval for the task; instead of "here are another three signatures" I was hoping the botop would find a wide range of similar linter errors that would likely be on the same page(s), and hit them all at once. As the run progressed, and new errors were found, they could just be added to the run logic without the need for a subsequent BRFA. If this is not the case, then it sure should be. Primefac (talk) 11:34, 31 January 2022 (UTC)[reply]

I have just been adding specific patterns to the replacement list as and when I find them. I don't want to use general purpose regexes to do replacements since this is a fully automated task. They work fine most of the time but edge cases are troublesome. My experience Linting Wikipedia has shown that people are... creative in using all sorts of things that would cause problems for general purpose regexes. Considering the size of this task, even with 99.9% accuracy, it would still leave thousands of false positives. This is the kind of bot task that when things go smoothly, most people wouldn't care, but if there are a few errors, lots of people would come to your talk with complaints. When the number of Lint errors are down to less than a hundred thousand instead 16.6 million today, then it would be possible to do a supervised run and try to clear out all errors in a page with a single edit. My current approach of using only specific replacements may not fix all errors in the page at a time, but it does the job by keeping false positives as close to zero as possible. This to me is the most important thing.

That said, I will increase the number of find and replace patterns the bot considers at a time so that more can be replaced if they are present in a page. The bot will take more time to process a page and will have to use a generic edit summary, but that's a good tradeoff I guess. ಮಲ್ನಾಡಾಚ್ ಕೊಂಕ್ಣೊ (talk) 18:07, 31 January 2022 (UTC)[reply]

That's not really a good reason to not consolidate all of these changes into one edit. If you have code that can correct <font> tags and you have another piece of code that can correct <tt> tags, then all you have to go is grab the wikitext, run it through the font tag code, then take the resulting corrected wikitext and run it through the tt tag code, then take the resulting wikitext and run it through any other blocks of delinting code that you have, and then finally write the changes to the article. Instead, you're grabbing the wikitext, running it through the font tag code, and then saving that edit. Then, sometime later, you're grabbing that wikitext again, and running it through tt tag code, and saving that edit. There's really no difference, except for the number of edits you're making. —⁠ScottyWong⁠— 07:14, 1 February 2022 (UTC)[reply]

To clarify, the code I have to correct font tags (i.e general purpose regexes to correct font tags) and some other Lint errors works fine most of the time, but gives some false positives which makes it not suitable for use in a fully automated task like this. You can read the first BRFA and this discussion for why I do not use such code with my bot. Usually in a situation like this, we would run it as semi-automated task and approve every edit before saving so that false positives can be discarded. But that is not possible here due to the huge number of pages involved. So I am left to work with a set of definite replacements, like specific user signatures and substituted templates, that are checked in a page before saving an edit. I have increased the number of replacements it will check to try and get more in an edit. This would be an example of when more than one of the replacements checked by the bot were present in a page and fixed in the same edit. ಮಲ್ನಾಡಾಚ್ ಕೊಂಕ್ಣೊ (talk) 15:58, 1 February 2022 (UTC)[reply]

it's not a COSMETICBOT violation as long the changes clear a maintenance error I disagree. The basis for COSMETICBOT in the first place is that cosmetic edits clutter page histories, watchlists, and/or the recent changes feed with edits that are not worth the time spent reviewing them, so they should not be performed unless there is an overriding reason to do so. That is not in the text of the policy, but clearly a balance has to be struck between usefulness and spamminess; the present case has fairly low usefulness (clearing a maintenance error is hardly high-priority).

Basically I agree with Scottywong. I have no strong feelings for or against COSMETICBOT, but if that bot task is deemed to be compliant, it means the policy is toothless, so we might as well remove it. (It might be argued that the bot task is against COSMETICBOT but should still be allowed as an explicit exception, but I do not see anyone making that argument.) Tigraan^{Click here for my talk page ("private" contact)} 13:08, 3 February 2022 (UTC)[reply]

"unless there is an overriding reason to do so" -- yes, and one of the common examples is right below in the policy text: "egregiously invalid HTML [..]". I mean, I agree that the policy sets no limit on spamminess, but that's a separate matter. —  HELLKNOWZ ∣ TALK 15:16, 3 February 2022 (UTC)[reply]

The policy says egregiously invalid HTML such as unclosed tags. Unclosed tags is a much more serious issue than deprecated tags, let alone deprecated tags that are still supported. "Egregiously invalid HTML" does not include only unclosed tags, but IMO it does not include font tags. At the very least, that is the sort of thing you would expect some discussion about at BRFA - if we are serious about enforcing the policy as written. Tigraan^{Click here for my talk page ("private" contact)} 13:13, 4 February 2022 (UTC)[reply]

Again I ask: When should we start fixing these deprecated tags, if not now? There are millions of instances of deprecated tags. Based on our historical pace of fixing Linter errors, it will take multiple years to fix all of them, especially since font tags in signatures are still, inexplicably, allowed by the MediaWiki software. – Jonesey95 (talk) 14:30, 4 February 2022 (UTC)[reply]

There is a saying that prevention is better than cure. Software updation is a natural part of all websites. That <font>...</font>, <center>...</center> and other obsolete tags are still supported doesn't mean it will continue to be so in future, hence why developers have marked them as errors and giving us time to replace them. Imagine logging in one day and seeing pages out of alignment and colors not being displayed, among other things. It had already happened once in July 2018. Will they not be egregiously invalid after that happens? These edits will benefit editors and readers by making sure that pages continue to display as editors intended when software changes. This is basically why COSMETICBOT allows fixing html even if it does not affect browsers' display or fixed before output by RemexHtml. My bot has already replaced about 2.5 million font tags while running the previous 10 BRFA Lint fixing tasks, hardly something there was no discussion about. ಮಲ್ನಾಡಾಚ್ ಕೊಂಕ್ಣೊ (talk) 14:36, 4 February 2022 (UTC)[reply]

(edit conflict) I consider tags that will stop working on par with unclosed tag pairs. Invalid markup is invalid. Be it for visual display, parsing, screen readers, printing, accessibility, forwards-compatibility, etc. —  HELLKNOWZ ∣ TALK 14:40, 4 February 2022 (UTC)[reply]

@Hellknowz: But these deprecated HTML4 tags are still supported, and no plans have been announced by anyone to stop supporting them at a specific date. Sure, eventually they will be unsupported, maybe next year, maybe coinciding with the heat death of the universe, or maybe sometime in between. At this point, we're reacting to something that we think might happen in the future, but we don't know when, and we don't know the specifics of how that deprecation will be handled. Maybe we'll be given 5 years notice of the deprecation. Maybe, by the time HTML4 tags are fully unsupported, we'll already be using HTML6 tags, and we'll have to go through this whole process a second time when we could have just done it once. The point is: we're acting reflexively but we don't know anything yet. And our response is to edit millions of decades-old closed AfDs to mess with someone's signature. I'm amazed that so many people are pushing back on this one. I mean, I can understand going through article space to fix these issues. But 15 year-old AfDs? Really? How is that worth anyone's time? What is the worst case scenario if someone happens to open a 15 year-old AfD and someone's signature is displaying in the default font instead of the custom font that the editor originally intended? —⁠ScottyWong⁠— 17:31, 4 February 2022 (UTC)[reply]

You say "worst case" but you describe one of the best cases. Worst case is the whole page throws a 5xx server error and doesn't render anything. I'm not saying it will happen, but I am saying we are trying to guess the future instead of fixing something that has been marked as deprecated. The original concern was that COSMETICBOT doesn't apply (or doesn't explicitly mention this use case). I only argue that it does follow policy even if people don't like it. But whether we actually want to fix this or leave it until it (may be) breaks is a different issue that wasn't opposed until watchlists lit up. This noticeboard can review approval on policy grounds, which I don't find a problem with. As I said, I don't like watchlist spam either and not doing all the replacements at the same time is pretty terrible. And this is likely not the first nor the last time something will need fixing. This sounds like a watchlist (search, sort, filter) functionality problem. —  HELLKNOWZ ∣ TALK 19:10, 4 February 2022 (UTC)[reply]

If some wikitext is causing 5xx errors, that's absolutely a developer/sysadmin problem, not one for on-wiki editors or bots to fix. Legoktm (talk) 06:42, 5 February 2022 (UTC)[reply]

• Some people have minor/bot edits showing on their watchlists for good reason. Eg, to monitor minor or bot edits to active pages. What they dont have them turned on for is to see a bot going back through a decades worth of archived/closed AFD's making trivial corrections to errors that barely deserve the name. Congratulations, you just pinged a load of deleted articles (quite a few contentious) back to the top of the watchlist of editors. Well done. Countdown to recreation in 5, 4, 3.... Only in death does duty end (talk) 15:14, 31 January 2022 (UTC)[reply]

• I'm coming here with a related problem. I was trying to search for something in the ANI archives, sorted by date. But that's impossible, because the date searched for is the last modified one, which is distorted because the bot has fixed minor errors long after the archive was created. For example, Wikipedia:Administrators' noticeboard/IncidentArchive969 contains two comments saying "please do not edit the archives", and yet this bot did it anyway. I don't really care what problem the bots were trying to solve, they have broken Wikipedia's search mechanism, making it unusable. Ritchie333 ^{(talk) (cont)} 12:16, 3 February 2022 (UTC)[reply]

  Honestly Wikipedia search is doomed. @Ritchie333: The problem you mention (searching archives where 'last modified' is updated by bot edits) I've gotten around by filtering by creation date, which should roughly correspond to the real date of entries in the archive. ProcrastinatingReader (talk) 15:21, 3 February 2022 (UTC)[reply]

  The problem Ritchie333 describes has existed forever, and it happens whether bots clean up the archive page, humans do manual tidying, or Xfd-processing editors do it. Pages need to be edited when MW code changes; that's just reality. It's the search dating that is broken. – Jonesey95 (talk) 15:32, 3 February 2022 (UTC)[reply]

  That doesn't have to be reality if we don't make it reality. We could choose to have our reality be one where archived discussion pages (like AN, ANI, XfD) are never edited, by bots or anyone else. And if the consequence is that, 20 years from now, an editor's fancy signature shows up in the default font instead of the custom font that the editor originally intended, well... we'll just have to find a way to emotionally deal with that problem. Coming from someone who uses custom fonts in their signature, I'm confident that I can find a way to work through that problem. It might take some extra therapy sessions, but I think I can do it. —⁠ScottyWong⁠— 17:36, 4 February 2022 (UTC)[reply]

• For what it's worth -- I think that this type of discussion often ends up with a pessimistic bent because people who don't see a problem don't care enough to comment about it -- I don't see a problem. Okay, maybe it breaks search: this seems like a potential real problem, but the deeper problem is that search sucks if it gets broken by this. I don't see why you would keep a page on your watchlist for ten years, besides the fact that ten years ago there wasn't a temporary-watchlist feature. It's not like there is any benefit to watchlisting an AfD that expired ten years ago -- unless there is? Is there? jp×g 20:12, 8 February 2022 (UTC)[reply]

Why not resolve this in MediaWiki?

I seem to recall seeing in some prior discussions (on

As opposed to clogging up the rendering pipeline by requiring translation being a massive waste of time and resources for every person viewing every revision everywhere? :) Revisions in a database are fundamentally cheap, anyway.

There was a brief time where one particular tag was translated, but it was quickly undone since it was used in places that were not compatible with the translation, among other reasons. Izno (talk) 04:40, 7 February 2022 (UTC)[reply]

Considering rendering is only an issue for pages that are edited regularly, and most of the pages with errors seem to be old/stale discussion pages, I'm not convinced mass bot edits is somehow better. —Locke Cole • t • c 05:34, 7 February 2022 (UTC)[reply]

No, the server needs to re-render every single viewed revision. The HTML for the revision of your comment is not stored as HTML, it is stored as wikitext. Izno (talk) 05:46, 7 February 2022 (UTC)[reply]

Locke Cole, the opposite of what you suggest is what actually happened. The code that renders pages had been silently correcting syntax errors for years, and when a new renderer was deployed some of those workarounds were not carried over. Hence Special:Linterrors, which flags errors that could cause rendering errors (most of which have been fixed by gnomes and bots since 2018) as well as conditions that will presumably have their workarounds removed at some point. For a deep dive, see mw:Parsing/Replacing Tidy. – Jonesey95 (talk) 04:52, 7 February 2022 (UTC)[reply]

@Jonesey95: Thank you for the pointer. What drew my attention to this was this edit which replaced <tt> with <code> tags which I thought was an odd thing to be done on a mass scale (semi-automated or not). —Locke Cole • t • c 05:45, 7 February 2022 (UTC)[reply]

The funny thing is that this doesn't even need to be fixed in MediaWiki, because no major browser in the world has trouble with HTML4 tags. If this bot didn't fix it, and MediaWiki software didn't fix it, then your browser would fix it. If anything, these issues should be fixed in article space only. I've still heard no legitimate reason why it's considered valuable to fix font tags on 12 year-old closed AfDs. —⁠ScottyWong⁠— 05:54, 7 February 2022 (UTC)[reply]

True, in the case of <tt>, which rabbit-holed me to this discussion, MDN still shows the tag as being fully supported in every major browser on the market. Being deprecated in the spec doesn't mean chase down issues that don't exist. —Locke Cole • t • c 06:11, 7 February 2022 (UTC)[reply]

I've still heard no legitimate reason is basically a fallacy, because I can say the same thing and have it be just as true.

As for browsers, that's true today. For the same reason MediaWiki developers can shut off an arbitrary tag, so too could the browsers. And they have done it before, which cannot be said of the developers.

Never mind that mobile today already applies a CSS reset to a bunch of the old tags—<tt> and <small> off the cuff, both of which render as normal text. Izno (talk) 06:15, 7 February 2022 (UTC)[reply]

Those are all great reasons to fix these deprecated tags within article space. However, what is the value in making millions of edits to fix these deprecated tags on old AfD pages that closed over a decade ago? If anyone can provide one good reason why that would be valuable, I'll gladly shut up. I don't think it's fallacious to ask "why?" and hope for a cogent answer. —⁠ScottyWong⁠— 17:12, 7 February 2022 (UTC)[reply]

I have, on multiple occasions, needed to check an old AFD log page (generally when someone improperly transcludes a

TFD

deletion template) and when there are linter errors on the page it takes ages to sort out where they're coming from in order for me to a) fix them, and b) find what I was originally looking for. To me, that is reason enough for to fix old things.

On the "watchlist spam" topic, I go through about once a year and clear out about half my watchlist of things that I will probably never need to see again, many of which are deletion discussion pages. Primefac (talk) 17:21, 7 February 2022 (UTC)[reply]

Bots Newsletter, January 2022

Graphs are unavailable due to technical issues. There is more info on Phabricator and on MediaWiki.org. BRFA activity by month Welcome to the ninth issue of the English Wikipedia's Bots Newsletter, your source for all things bot . Vicious bot-on-bot edit warring... superseded tasks... policy proposals... these stories, and more, are brought to you by Wikipedia's most distinguished newsletter about bots. After a long hiatus between August 2019 and December 2021, there's quite a bit of ground to cover. Due to the vastness, I decided in December to split the coverage up into a few installments that covered six months each. Some people thought this was a good idea, since covering an entire year in a single issue would make it unmanageably large. Others thought this was stupid, since they were getting talk page messages about crap from almost three years ago. Ultimately, the question of whether each issue covers six months or a year is only relevant for a couple more of them, and then the problem will be behind us forever. Of course, you can also look on the bright side – we are making progress, and this issue will only be about crap from almost two years ago. Today we will pick up where we left off in December, and go through the first half of 2020. Overall In the first half of 2020, there were 71 BRFAs . Of these, Y 59 were approved, and 12 were unsuccessful (with N2 8 denied, ? 2 withdrawn, and 2 expired). January 2020 Yeah, you're not gonna be able to get away with this anymore. Y ST47Bot, Seppi333Bot, Qbugbot 4 · N2 PkbwcgsBot 25, PkbwcgsBot 26 A new Python 2.7 would be removed in coming updates. Toolforge itself planned to drop Python 2 support in 2022. Two new bots dropped in January 2020: ST47Bot, the successor to STBot, and Seppi333Bot, whose task was to perform a weekly update of four "massive" wikitables for WikiProject Molecular Biology. CfD 'd. February 2020 Y AnomieBOT 78, Cewbot 2, DannyS712 bot 67, DannyS712 bot III 63, PearBOT 6, DemonDays64 Bot 2 · N2 PearBOT On February 1, some concerns were raised about ListeriaBot performing "nonsense" edits. Semi-active operator Magnus Manske (who originally coded the Phase II software|precursor of MediaWiki) was pinged. Meanwhile, the bot was temporarily blocked for several hours until the issue was diagnosed and resolved. On February 18, Wikipedia:Sockpuppet investigations/Cases/Overview . March 2020 Y Bitbotje, FACBot 6, TheSandBot 8, Qbugbot 5, Cewbot 3, SDZeroBot 2, PrimeBOT 31, WikiCleanerBot 12 · AntiCompositeBot In March, a long Skdb about the troubling trend of bots "expiring" without explanation after their owners became inactive. This can happen for a variety of reasons -- API changes break code, hosting providers' software updates break code, hosting accounts lapse, software changes make bots' edits unnecessary, and policy changes make bots' edits unwanted. The most promising solution seemed to be Toolforge hosting (although it has some problems of its own, like the occasional necessity of refactoring code). Some of the twelve bot tasks approved this month were draftspace ). April 2020 Y TheSandBot 9, BHGbot 5, DannyS712 bot 69, SDZeroBot 3, ST47ProxyBot, WugBot 4, SDZeroBot 4, DatBot 10, WikiCleanerBot 11, WikiCleanerBot 13, WikiCleanerBot 14, AntiCompositeBot 2, Cewbot 4 · N2 DaedanBot 1 · Creffbot A non-free images to others). Some said it was doing good work, and others said it was operating beyond its remit. It was blocked on April 10; the next day it was unblocked, reblocked from article space, reblocked "for specified non-editing actions", unblocked, and indeffed. The next week, several safeguards were implemented in its code by Magnus; the bot was allowed to roam free once more on April 18. diffusing categories), TheSandBot 9 (taking over some tasks from the inactive RonBot), and WugBot 4 (updating data at {{Interactive COVID-19 maps}}). Also, DatBot 10 was approved to scale down non-free SVGs — I didn't even know that was a thing. A WP:BOTCOMM should explicitly specify that bot operators must be responsive to concerns raised on English Wikipedia specifically (as opposed to Phabricator, SourceForge, Toolforge, et cetera). Eventually, the policy was amended to its current form : Issues and enquiries are typically expected to be handled on the English Wikipedia. Pages reachable via anonymity ) can supplement on-wiki communication, but do not replace it. May 2020 Y MajavahBot 3, DannyS712 bot 70, CitationCleanerBot 5, Dreamy Jazz Bot 4, QEDKbot, William Avery Bot 2, DannyS712 bot 68, PrimeBOT 32, TheSandBot 10, Bot1058 7, BHGbot 6, EaglesBot 2, Cewbot 5, WikiCleanerBot 17 · N2 MDanielsBot, PhuzBot 3, Gedimon · ? PearBOT 7 MajavahBot 3, an impressively meta bot task, was approved this month for maintaining a list of bots running on the English Wikipedia. The page, located at User:MajavahBot/Bot status report, is updated every 24 hours; it contains a list of all accounts with the bot flag, as well as their operator, edit count, last activity date, last edit date, last logged action date, user groups and block status. Other approvals for this month were William Avery Bot 2 (processing redirects in {{R from scientific name}}), DannyS712 bot 68 (allowing users to schedule bot reminders for themselves), TheSandBot 10 (removing blocked/locked users from Category:User talk pages with conflict of interest notices), Bot1058 7 (updating Wikipedia:Requested moves/Technical requests/Permalink), Dreamy Jazz Bot 4 (adding {{WikiProject Biography|living=no}} to talk pages where their associated article is in a year of death category), and QEDKbot (deleting and nominating empty categories). General syntax fixing tasks were approved for DannyS712 bot 70, CitationCleanerBot 5, PrimeBOT 32, BHGbot 6, Cewbot 5, and EaglesBot 2, as well as WikiCleanerBot 15, 16, and 17. Seven bots who fell below activity requirements (with neither the bots nor their operators having any edits or logged actions in two years) had their bot flags removed. They were NekoBot, Robert SkyBot, DASHBotAV, VoxelBot, JackieBot, ReferenceBot, and CensusBot. On May 12, EmausBot was blocked. June 2020 Y QEDKbot 2, AnomieBOT 79, WOSlinkerBot 6, Yapperbot 2, Yapperbot, AnomieBOT 80, Dreamy Jazz Bot 5, WikiCleanerBot 18, HasteurBot 15, MusikBot II 4, NoSandboxesHere · N2 RedWarn · ? Seppi333Bot 2 In July 2017, partial-block of Legobot for the dashboard was proposed. Some months later, on June 16, Headbomb said: "A full block serves nothing. A partial block solves all current issues [...] Just fucking do it. It's been 3 years now." The next day, however, Legoktm disabled the task, and the dashboard was successfully refactored. On June 7, LTA had "weaponized" the bot to harass editors). David Tornheim opened a discussion about whether bots based on closed-source code should be permitted, and proposed that they not. He cited a recent case in which a maintainer had said "I can only suppose that the code that is available on GitHub is not the actual code that was running on [the bot]". Some disagreed: Naypta said that "I like free software as much as the next person, and I strongly believe that bot operators should make their bot code public, but I don't think it should be that they must do so". Two new bots had their first BRFAs approved: Feedback Request Service, and NoSandboxesHere, to remove {{user sandbox}} from articles in draftspace using Category:Non-userspace pages using User sandbox . Conclusion What's next for our intrepid band of coders, maintainers and approvers? Will Citation bot ever be set free to roam the project? What's the deal with all those book links that InternetArchiveBot is adding to articles? Should we keep using Gerrit for MediaWiki? What if we had a day for bots to make cosmetic edits? These questions will be answered — and new questions raised — by the February 2022 Bots Newsletter. Tune in, or miss out! Signing off... jp×g 23:22, 31 January 2022 (UTC)[reply] (You can subscribe or unsubscribe from future newsletters by adding or removing your name from this list.)

It looks like we have a few inactive bots by definition (both account and operator haven't edited in 2 years) according to the Majavah report. (

Pending removals

• Detroiterbot run by MJCdetroit
• FeedBot run by Feedintm
• GrashoofdBot run by Grashoofd
• DpmukBOT run by Dpmuk
• Image-req-proj-bot run by Traveler100
• FPBot run by FunPika
• PeerReviewBot run by CBM
• ExpertIdeasBot run by I.yeckehzaare
• タチコマ robot run by とある白い猫
• Theo's Little Bot run by Theopolisme
• Wiki Feed Bot run by Fako85
• Fz29bot run by Fz-29
• KadaneBot run by Kadane

Required notifications

All operators notified on their talk pages. — xaosflux ^Talk 14:22, 1 February 2022 (UTC)[reply]

• •  Removed all removed following this notice and no response from the operators. — xaosflux ^Talk 21:51, 11 February 2022 (UTC)[reply]

Forward outlook

And a few more in a month or so:

• PDFbot run by Dispenser
• Makecat-bot run by Makecat
• TAP Bot run by Thine Antique Pen
• BG19bot run by Bgwhite
• ProteinBoxBot (blocked also) run by Andrew Su and Julialturner
• TheMagikBOT run by TheMagikCow

Long inactives but not outside of policy

We should consider asking the active bot operators whether all the rest of the bot accounts that haven't edited since a while ago (pick a number, 5 years seems fine generally) still need a bot flag. These particularly stand out...:

• SprinterBot is a redirect to RscprinterBot

  Operator talk left to see if this can be retired. — xaosflux ^Talk 14:25, 1 February 2022 (UTC)[reply]

   Removed per operator request in Special:Diff/1069313440. — xaosflux ^Talk 18:02, 1 February 2022 (UTC)[reply]

• CeraBot is a redirect to Cerabot~enwiki

  Operator talk left to see if this can be retired. — xaosflux ^Talk 14:27, 1 February 2022 (UTC)[reply]

• Citation bot 1, Citation bot 2, Citation bot 3, Citation bot 4 subsumed? by Citation bot
• Helpful Pixie Bot and Femto Bot (operator presently blocked)
• Legobot II (blocked)
  • Feel free to remove the bot flag. Legoktm (talk) 17:58, 4 February 2022 (UTC)[reply]
• TohaomgBot (blocked)
• KasparBot (blocked)
• FRadical Bot (blocked, and the operator CU-blocked)

   Removed — xaosflux ^Talk 10:42, 1 February 2022 (UTC)[reply]

• Cydebot (blocked)

Discussion

These ones seem like low-hanging fruit, but I think the rest should be queried as well. Izno (talk) 05:30, 1 February 2022 (UTC)[reply]

This was actually the subject of a proposal I was about to make -- I think it might be useful for a bot task that monitored the on-wiki activity of bot maintainers. Either "last edit date", or "edits in last 180 days", or something along those lines -- this could either go on a central status page or on the infobox of a bot's userpage. Oftentimes, the writers and maintainers of bots will go inactive for a long time, which makes it a little discouraging to report bugs / suggest new features, as it's likely they will fall into a black hole. Sjp×g 07:45, 1 February 2022 (UTC)[reply]

This seems like a good idea, but relies on something we don't actually have: a programmatic listing of all bot:operator relationships! I'd actually like to see a page/table for this. (which could then also possible be bot-updated with the last edit/action dates of each periodically) - but bootstrapping it takes some effort. — xaosflux ^Talk 10:53, 1 February 2022 (UTC)[reply]

Picking a couple recently-approved BRFAs at random, it looks like "Operator:" is formatted relatively consistently between 2013 and now -- I might be able to scrape through these and come up with something. Maybe something that had to be manually verified, but it'd be something. jp×g 11:36, 1 February 2022 (UTC)[reply]

Hell, even this one from 2007 has the operator noted the same way. jp×g 11:38, 1 February 2022 (UTC)[reply]

Here's an idea, your idea might be better though: There's bot userboxes that are posted on the owner's page and indicate the operated bot as one of the parameters. Example: {{Template:User bot owner|Acebot}}. There's probably userboxes or templates for the bot's page too. These templates also populate categories such as Category:Wikipedia bot operators. –Novem Linguae (talk) 11:43, 1 February 2022 (UTC)[reply]

It's only somewhat related but I'd like to see a list of active approved bot tasks. There are lots of pages in Category:Approved Wikipedia bot requests for approval but this includes tasks that finished, op went inactive, are obsolete, consensus changed, bot went down, etc...

A useful way would be for the closing BAG to add a characteristic of a task (eg the task # in edit/action summaries) to Wikipedia:Bot activity monitor going forward, which would achieve that. Then maybe old tasks can be added on an ad-hoc basis. ProcrastinatingReader (talk) 12:40, 1 February 2022 (UTC)[reply]

It would likely be a "going forward" for now, with potential retroactive categorisation later, but we could always base it on the "Edit period(s)" section of the task - if it's a One-Time-Run, sub-categorise it as such, otherwise put it into some sort of "ongoing" subcat. Primefac (talk) 20:53, 1 February 2022 (UTC)[reply]

• I left operator notices for the first batch, if no response in a week we will deflag them and mark as {{retired|bot=yes}}. — xaosflux ^Talk 10:47, 1 February 2022 (UTC)[reply]

• SprinterBot, I believe there is little realistic prospect of resurrecting this bot's tasks, so feel free to deflag the account. Rcsprinter123 (comment) 17:32, 1 February 2022 (UTC)[reply
  ]

I added a column to

talk!) 11:05, 1 February 2022 (UTC)[reply

]