Commons:Village pump/Technical

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Shortcuts: COM:VP/T • COM:VPT

Welcome to the Village pump technical section
Technical discussion
Village pump/Technical
 Bug reports
 Code review
Tools
 Tools/Directory
 Idea Lab



This page is used for technical questions relating to the tools, gadgets, or other technical issues about Commons; it is distinguished from the main Village pump, which handles community-wide discussion of all kinds. The page may also be used to advertise significant discussions taking place elsewhere, such as on the talk page of a Commons policy. Recent sections with no replies for 30 days and sections tagged with {{Section resolved|1=--~~~~}} may be archived; for old discussions, see the archives; recent archives: /Archive/2024/05 /Archive/2024/06.

Please note
 
SpBot archives all sections tagged with {{Section resolved|1=~~~~}} after 1 day and sections whose most recent comment is older than 30 days.

New tool for cropping and rotating images (proposal)

[edit]

MediaWiki or at least Commons should have a function that allows to crop images and save them as separate ones directly, including the original description.

Why would we need it when most phones can crop or even rotate images?

  • preserve image resolution
  • preserve metadata
  • ensure derivation is annotated on original file

We currently have a bot and a user tool that have some of the functionality, but this lags behind a reasonable functionality. Maybe WMF development can be facilitated by adopting some of their code. Enhancing999 (talk) 08:13, 12 April 2024 (UTC)[reply]

Enhancing999, CropTool has no lag for me. You ask to crop an image, you crop it and it is immediately uploaded. I use it regularly and I do not have any complains about it. Rotatebot also seems to work fine. You tag a file for rotating and within 24 hours it is rotated. I guess we could have a tool to rotate immediately, but the current approach is not bad. --Jarekt (talk) 00:42, 18 April 2024 (UTC)[reply]
There are two years of open bug reports and feature requests on the tools talk page.
I agree it works on files of 500 KB, but larger ones fail now, especially when attempt to fix rotation by a few degrees. Rotatebot can't handle that. Also the automated border detection fails. Enhancing999 (talk) 08:15, 18 April 2024 (UTC)[reply]
Lots of miscellaneous failures over the last few years, and no one committed to maintaining it.
I'd be glad to see a group of three or more volunteers take it on, or WMF, but finding one more individual to do it just postpones collapse, because the environment keeps changing. - Jmabel ! talk 12:46, 18 April 2024 (UTC)[reply]
Isn't development of key features meant to be done by paid WMF staff? It's not a content/editorial issue. Enhancing999 (talk) 07:41, 20 April 2024 (UTC)[reply]
@Enhancing999:
Why would we need it when most phones can crop or even rotate images?
That assumes that users use phones. The section ‘Mobile versions of user talk pages are useless’ did point out that some users exclusively use phones, shunning other devices such as laptops and desktop computers.
But not all users are like that. There are good reasons for using other devices, including (but not limited to) the criticisms of Wikimedia's mobile versions given in that same section. Brianjd (talk) 08:30, 23 April 2024 (UTC)[reply]
This section is actually about two distinct actions:
  1. Edit an image (by cropping or rotating), while otherwise preserving image quality and metadata.
  2. Upload the edited image with an appropriate filename and description and ensure that the original image’s description is updated accordingly.
But this section proposes to bundle these actions into one tool.
The problem is that there are other important types of edits besides crops and rotations. Even if Commons can’t help with the edits themselves, it should still help with filenames and descriptions, the same way as it does for crops and rotations. Brianjd (talk) 08:46, 23 April 2024 (UTC)[reply]
Well, in the past, bundling these actions have been proven fairly helpful. If you think other tools are needed at Commons, please discuss this elsewhere. Enhancing999 (talk) 14:35, 23 April 2024 (UTC)[reply]
@Enhancing999: I agree that bundling these actions into one tool is helpful. But that one tool should be implemented as a wrapper around more specific tools (one for cropping and one for handling filenames and descriptions), which should also be accessible separately. Brianjd (talk) 08:21, 24 April 2024 (UTC)[reply]
As user, I'm mainly interested in a stable, working tool. For the rest, WMF pays IT staff. Enhancing999 (talk) 08:40, 24 April 2024 (UTC)[reply]
There has been some code written to include cropping into mediawiki. Just needs someone to take it over the line. Bawolff (talk) 21:48, 11 May 2024 (UTC)[reply]
This talk page reports several problems with the current version of CropTool. As a user, I am indifferent about whether it is repaired or replaced. But something must be done, urgently. Pommée (talk) 08:17, 19 May 2024 (UTC)[reply]

Tech News: 2024-20

[edit]

MediaWiki message delivery 23:55, 13 May 2024 (UTC)[reply]

HotCat

[edit]

I've found my HotCat not really working today, it doesn't highlight any images after I select All. No Swan So Fine (talk) 10:43, 15 May 2024 (UTC)[reply]

Are you sure you mean HotCat and not Cat-a-Lot ? —TheDJ (talkcontribs) 11:23, 15 May 2024 (UTC)[reply]
You are correct, sorry. I meant Cat-a-Lot! I've replied at [11] No Swan So Fine (talk) 11:53, 15 May 2024 (UTC)[reply]

Changes to upload by URL

[edit]

As of 1330 UTC today, there have been some backend changes to how uploading files by URL works. In order to address historical issues where larger files time out during upload (and to hopefully enable us to increase upload limits in future), URL uploads are now performed in the background after submission. This has been successfully tested in a few environments, but if you notice any issues with Special:Upload when using URLs, please post in the relevant Phabricator task. HNowlan (WMF) (talk) 17:02, 16 May 2024 (UTC)[reply]

Tech News: 2024-21

[edit]

MediaWiki message delivery 23:01, 20 May 2024 (UTC)[reply]

Javascript issue

[edit]

Hi, I got a JS problem. Help:Gadget-DelReqHandler doesn't load or work. Broken with Monobook and Timeless on Chrome and Firefox on Windows 10. Works with other skins. Yann (talk) 10:45, 23 May 2024 (UTC)[reply]

 Info -> Commons:Administrators'_noticeboard#Problem_with_DelReqHandler.--Wdwd (talk) 12:36, 23 May 2024 (UTC)[reply]

Is it possible to search EXIF tags?

[edit]

Is it possible to search by EXIF tag, i.e., file metadata? The full text search allows a variety of operators, but I don't see anything for this, and it would be useful for some of the work I'm doing. grendel|khan 12:34, 25 May 2024 (UTC)[reply]

https://quarry.wmcloud.org/ has some limited support for searching through metadata, but it is far from ideal. Bawolff (talk) 23:13, 26 May 2024 (UTC)[reply]

Replicating s:Module:Copyright-until here

[edit]

I am hoping that someone can do some work with s:Template:Copyright-until and the module by user:CalendulaAsteraceae at Wikisource to help reduce some of the problematic error generation that comes from just pasting links to files on a category page. Those cat pages are constantly generating errors for the red links, and it would be good if we could migrated to something a little less problematic. Thanks.  — billinghurst sDrewth 11:58, 27 May 2024 (UTC)[reply]

Tech News: 2024-22

[edit]

MediaWiki message delivery 00:12, 28 May 2024 (UTC)[reply]

Can someone perhaps remove all the deleted and renamed files from User:Dispenser/Double extension so that only the ones that still need to be fixed are left? Jonteemil (talk) 20:46, 28 May 2024 (UTC)[reply]

Anyone can do so by forking the Quarry query mentioned in the SQL snippet (look for Fork in the upper right corner of the Quarry page), but I’m not comfortable editing a random other user’s subpage without seeing an explicit permission for doing so. —Tacsipacsi (talk) 17:37, 9 June 2024 (UTC)[reply]
I tried forking it, see quarry:query/83409, but it just keeps running. It has run for a week now, yet still running. Jonteemil (talk) 17:21, 11 June 2024 (UTC)[reply]

Searching with 'haswbstatement' doesn't seem to filter properly.

[edit]

See here for an example. I'm searching in the File namespace for incategory:"Photos by Grendelkhan" -haswbstatement:P170, which brings up files which do have creator (P170) defined on them. If I switch to incategory:"Photos by Grendelkhan" haswbstatement:P170, I get zero results. I'm confused. According to mw:Help:Extension:WikibaseCirrusSearch, this should be filtering on the presence of a structured data field, but if anything it's doing the opposite? grendel|khan 15:18, 30 May 2024 (UTC)[reply]

as i'm writing:
  1. Media in category "Photos by Grendelkhan" The following 200 files are in this category, out of 725 total.
  2. incategory:"Photos by Grendelkhan" -haswbstatement:P170 -> Results 1 – 30 of 725
  3. 725-725=0 = incategory:"Photos by Grendelkhan" haswbstatement:P170
RZuo (talk) 21:26, 30 May 2024 (UTC)[reply]
Exactly; all of those photos have creator (P170) statements on them, but the filtering isn't doing the right thing. grendel|khan 16:44, 3 June 2024 (UTC)[reply]

Tech News: 2024-23

[edit]

MediaWiki message delivery 22:32, 3 June 2024 (UTC)[reply]

You are invited to join the discussion about localization of the Cat-a-lot gadget at MediaWiki talk:Gadget-Cat-a-lot.js#MediaWiki:Mobile-frontend-return-to-page vs MediaWiki:Returnto. —⁠andrybak (talk) 21:59, 4 June 2024 (UTC)[reply]

Enable numeric sorting for Commons categories

[edit]

English Wikipedia has numeric sorting for its categoesolveries, e.g. en:Category:Companies by year of establishment and country see that "0–9" that is created to as MediaWiki understands these are numbers and should be sorted using numeric sorting rather than character by character but Wikimedia Commons doesn't do that sorting and e.g. there is no "0-9" on Category:Photographs_in_the_Golestan_Palace_Library so 1039 is put before 108 I've proposed that on phab:T366703 but I'm told to discuss that with Commons community also. English Wikipedia uses the same config and the change from going from uppercase sorting to uca-default is discussed at phab:T136150. −ebrahimtalk 13:29, 5 June 2024 (UTC)[reply]

Obsolete, previously agreed to have, Commons:Village_pump/Archive/2024/04#h-Numerical_sorting_in_categories-20240401232200ebrahimtalk 17:24, 5 June 2024 (UTC)[reply]

Subcategory sorting broken

[edit]

Did this get changed today?
Any idea why "M" for subcategories is between "A" and "C" on Category:Goethe streets ?
Also "next" on Category:Non-empty category redirects doesn't work anymore. Enhancing999 (talk) 14:38, 6 June 2024 (UTC)[reply]

@Enhancing999: Yes, according to phab:T362494 it seems to have been deployed about half an hour ago. I suspect that there may be some transient problems while sort keys get updated for the new ordering. --bjh21 (talk) 14:45, 6 June 2024 (UTC)[reply]
Weird. Let's given them a week to sort it out. Enhancing999 (talk) 14:51, 6 June 2024 (UTC)[reply]
Seems they gave up on it. Phab:T366809. [25] Enhancing999 (talk) 15:38, 6 June 2024 (UTC)[reply]

The find from the post-mortem at phab:T362494 seems to be that the script was run with the wrong options leading to change every sortkey, not just numeric ones. That it takes a long time to run shouldn't be that much an issue. Enhancing999 (talk) 09:15, 7 June 2024 (UTC)[reply]

@Enhancing999 That's not entirely correct. The problem is that the category collation script wasn't run at all, and running it would take weeks to months, during which categories would be broken in the ways you noticed. Because of the size of the Commons categorylinks database, this would cause more stress to the database than the database administrators are willing to tolerate. Using pure numeric collation instead of uca-default-u-kn would reduce the broken categories issue, but wouldn't change the database load issue. The size of the Commons database (specifically the tables for categories and templates) is so large that on its own it is causing stability issues (phab:T343131). We will need to take a hard and careful look at how we use categories and templates and reduce unnecessary uses of both, migrating data to SDC (which is stored more efficiently) where reasonable. Unfortunately we won't be able to do this on our own, it will need WMF support. AntiCompositeNumber (talk) 00:10, 10 June 2024 (UTC)[reply]
The two problems I reported here seem to be due entirely due to the wrong options being used. Or am I missing something? Enhancing999 (talk) 08:41, 10 June 2024 (UTC)[reply]

Cat-a-lot still doesn't work well

[edit]

Though this section was archived because the problem would supposedly have been resolved, Cat-a-lot still doesn't work for subcategories with at least one subcategory. So can this problem get a high priority and really be solved? It is now almost half a year that it arose. JopkeB (talk) 17:04, 6 June 2024 (UTC)[reply]

Also, could it be made to work on Special:MediaSearch? Or fix Special:Search to display the full images? Enhancing999 (talk) 09:12, 7 June 2024 (UTC)[reply]

sort

[edit]

Any ideas about how to sort files in subcategories of Category:Photographs in the Golestan Palace Library by number? each subcategory might contain more than 100 files in the future. Hanooz 12:46, 7 June 2024 (UTC)[reply]

Either
  • by setting the album category with sortkey in a template
  • or by adding a sortkey in on the file sample.
Enhancing999 (talk) 13:33, 7 June 2024 (UTC)[reply]
Thanks. I'm looking for a more proper way as there are more than 3000 files like the one you mentioned. Hanooz 13:40, 7 June 2024 (UTC)[reply]
It should be fairly straightforward either way. Commons:Bots/Work_requests can help. Enhancing999 (talk) 13:49, 7 June 2024 (UTC)[reply]
User:Hanooz: This was requested at Commons:Village pump/Archive/2024/04#h-Numerical sorting in categories-20240401232200 and now is purposed and blocked on phabricator:T362494 and interestingly enough my involvement with the issue is because of the same category also. If this didn't happen we can prepend 0 to subcategories sortkey as a work around. −ebrahimtalk 14:26, 10 June 2024 (UTC)[reply]
I guess a three-digit sortkey must be added to each file. Hanooz 18:35, 10 June 2024 (UTC)[reply]

Special:Search and broken thumbnails

[edit]

Apparently this was fixed 2 years back, but somehow came back: phab:T320459 "Should not see squared, cropped thumbnails next to search results for files on Special:Search page on Commons".

What has changed since? How can we fix it again? @El Grafo, Matthiasmullie, and Raymond: Enhancing999 (talk) 10:53, 10 June 2024 (UTC)[reply]

Might have to do with the recent changes to MediaWiki:Common.css – @Ebrahim: could you check that please? El Grafo (talk) 12:23, 10 June 2024 (UTC)[reply]
Enhancing999, El Grafo: This isn't related to my change as that was about jQuery UI dialog in RTL and the code is available in several wikis so the change is MediaWiki side as far as I can tell but surely I want to help to fix it but I don't remember what the fixed situation looked like but have made this temporarily link so you can see if this is what you want which if so I can apply to be shown always −ebrahimtalk 14:40, 10 June 2024 (UTC)[reply]
@ebrahim: That would be an improvement. The images are no longer randomly cropped. Ideally, I guess lines for each result would have a similar height and the text would still align even if the images vary in actual width. A bit like the sample above (obviously with a larger text area). Thanks for looking into this. Enhancing999 (talk) 15:20, 10 June 2024 (UTC)[reply]
Enhancing999: I've applied the change I shown on the temporarily link but I don't know yet how to apply this one you've brought which indeed looks great −ebrahimtalk 15:52, 10 June 2024 (UTC)[reply]
Is there a way to revert or toggle this back to the squares or set a maximum height? The changes have resulted in vertically large images being huge and taking up several screenfuls of screen space. This is making using search to scan many images at once nigh on impossible. It is excruciating. Gnomingstuff (talk) 16:56, 10 June 2024 (UTC)[reply]
For a specific example of what I mean, search "Unicode Tangut 17000 to 17FFF". If an image with similar proportions shows up in search -- and they frequently do, even in my experience in a short span of time -- image search becomes massively, massively cumbersome. Gnomingstuff (talk) 16:58, 10 June 2024 (UTC)[reply]
I made some MediaWiki core changes to get rid of the table layout. It might be that it is related to that ? —TheDJ (talkcontribs) 18:50, 10 June 2024 (UTC)[reply]
Ok, this is definitely related. I wasn't aware that Commons had this override, so i couldn't take it into account when I was working on this. I have one small problem, in that I don't remember what sizing this used exactly before. I will test this out with my local MediaWiki install tomorrow and can then give you update CSS code for Commons to fix the problem. —TheDJ (talkcontribs) 21:11, 10 June 2024 (UTC)[reply]
Hmm, this was already broken for quite a bit before my change (ever since the new image html was in place like a year ago, although my change made it break harder). OK, what is it that we WANT ? I think we want consistent indentation of the text. Core currently specifies the image as 90px wide. And then we want portrait images to not get cut off. But we also need portrait to be limited, or some exceptional cases would span half the page. Let's say limit height to a maximum of 180px ? Then we want to preserve aspect ratio. We currently vertically align to the top, and horizontally centre. We could align left instead, that might visually look more consistent. Is that a good representation ?
There are some indications that Commons was using more than 90px wide... Does anyone know if Commons was using a larger image in the search results than other wikis ? —TheDJ (talkcontribs) 19:15, 11 June 2024 (UTC)[reply]
2021
2017
Ideally we would use a size that already exists, i.e. the thumbnail needn't be generate for the search.
Above screenshots from 2017 and 2021. I doubt that was 90px. Not sure if anything is even visible on 90px.
Categories use 120px, maybe a bit larger would be fine. Enhancing999 (talk) 19:30, 11 June 2024 (UTC)[reply]
Gnomingstuff: Your take is also totally understandable but this change is hard to revert for a specific user so consider my change temporarily till we see what User:TheDJ can come up with as honestly I don't remember how it looked like before the changes. −ebrahimtalk 06:53, 11 June 2024 (UTC)[reply]
@Ebrahim admittedly I am not a CSS expert but max-height on the container and object-fit on the image (to crop anything that falls outside the container)? Gnomingstuff (talk) 14:05, 11 June 2024 (UTC)[reply]
The idea is to avoid random cropping (who could possible want that). It really makes search results hard to read. Enhancing999 (talk) 14:08, 11 June 2024 (UTC)[reply]

Tech News: 2024-24

[edit]

MediaWiki message delivery 20:17, 10 June 2024 (UTC)[reply]

New designs for logo detection tool

[edit]
Mockup for an alert when a logo is detected

Hello all! We're happy to share that we will work on logo detection in the following months and that we defined an initial approach for this.

You can read more at the project page and you can have your say in the project's talk.

We want your feedback on it, and we need your insights on how to further tune the detection tool.

Thanks for your attention! Sannita (WMF) (talk) 13:55, 11 June 2024 (UTC)[reply]