Topic on Project:Support desk

Apparently MediaWiki is using the popen() function PHP. A lot of users are on shared hosting packages. Including me. How is such a thing possible, when popen() is an actual execution command, open to abuse in the wrong hands? (See: http://stackoverflow.com/questions/3115559/exploitable-php-functions)

Maybe MediaWiki is sticking with it, maybe not. What's worse is that there's absolutely no mention of this change anywhere in the change log, which is absurd. Is there a short term solution I can use? Downgrade, force MW to use the old file upload system? Anything?

All help would be appreciated.

If there's "absolutely no mention of this change anywhere in the change log" I wonder which ChangeLog(s) you checked, because popen has been in the code for a long time. At least long enough that https://git.wikimedia.org/blame/mediawiki%2Fcore.git/HEAD/includes%2FGlobalFunctions.php and https://git.wikimedia.org/blame/mediawiki%2Fcore.git/HEAD/includes%2Ffilebackend%2FFSFileBackend.php do not list the specific commits when it was introduced, once upon a time.

https://git.wikimedia.org/blob/mediawiki%2Fcore.git/REL1_23/RELEASE-NOTES-1.23 which is referenced in http://www.mediawiki.org/wiki/MediaWiki_1.23. File uploads were working beautifully in 1.22.7. My hosting provider hasn't changed anything, so what could be the problem?

The problem could be that some area of code received another popen call() and that area of code is triggered more often than those areas that already had popen() calls which did not create problems for you. I guess.

Hmm, the first use of popen was in REL1_3 in GlobalFunctions.php: https://github.com/wikimedia/mediawiki-core/blob/REL1_3/includes/GlobalFunctions.php

The FSFileBackend (since REL1_20) was created with the use of popen: https://github.com/wikimedia/mediawiki-core/blob/REL1_20/includes/filebackend/FSFileBackend.php

So, if you have an error message, so wen can see, in which file the popen fails, then we can look into it, when this was introduced, but i think it was before 1.22.

Warning: popen() has been disabled for security reasons in /nfs/home/***/domains/*.com/public_html/includes/filebackend/FSFileBackend.php on line 702

Warning: stream_get_contents() expects parameter 1 to be resource, null given in /nfs/home/***/domains/*.com/public_html/includes/filebackend/FSFileBackend.php on line 709

Warning: fclose() expects parameter 1 to be resource, null given in /nfs/home/***/domains/*.com/public_html/includes/filebackend/FSFileBackend.php on line 710

Warning: cp '/tmp/phpSb0DaE' '/nfs/home/***/domains/*.com/public_html/images/7/7c/Image-e-14124d.png' in /nfs/home/deb31188n3/domains/dmowiki.com/public_html/includes/filebackend/FSFileBackend.php on line 282

http://www.mediawiki.org/wiki/Project:Support_desk#Upload_system_differences_between_1.22.7_and_1.23.0_43943

I have the same problem. I am able to upload on 1.22.7 but not on 1.23.0

nothing changed on my hoster (and popen is disabled, but works on 1.22.7)

EDIT: I restored 1.22.7 backup and I am trying to figure the error/changes out

This post was posted by Raid~mediawikiwiki, but signed as Raid.

Same. Restored the 1.22.7 backup, works fine. Lost a bit of data, but just a bit. Uploading now works fine: 17:49, 10 June 2014 Thumbnail for version as of 17:49, 10 June 2014 1,920 × 1,080 (791 KB)

Your issue is caused by PHP's safe_mode, a broken-by-design setting that does more harm than good. Unfortunately it seems that some people just don't bother reading manuals and enable that setting because "hey, it has 'safe' in its name, so clearly it must be good!". The PHP manual states that safe_mode has been "DEPRECATED as of PHP 5.3.0 and REMOVED as of PHP 5.4.0" (finally!).

I suggest you contact your hosting provider and provide them that PHP.net link and ask to have safe_mode disabled (and/or PHP upgraded to 5.4.0 or newer; either one works).

safe_mode is off.

Safe_mode is Off here too

On my shared host I tried the following php version (I can select on CP): 5.5.9 5.4.23 5.3.28

and mw 1.23.0 upload system is not working at all.

mw 1.22.7 is working perfectly

This post was posted by Raid~mediawikiwiki, but signed as Raid.

What about "disable_functions" in php.ini? Is popen or proc_open listed there? Does the problem still happen after removing it?

You are all on shared hosts, right? You can see disable_functions in phpinfo(), example: test.php:

<?php
echo phpinfo();
?>

We have a customer on one of or servers and he runs in the same issue, php version is 5.4.* and safemode flag is off and checked apache vhost file manually for it.

Regards,

What about disable_functions, as asked before?

disable_functions: system,popen,dl,passthru,proc_open,shell_exec

I am on a shared linux host (aruba.it), currently I am running PHP 5.4.23.

Mw 1.22.7 upload system is working well. Mw 1.23.0 isn't

if a overwrite the mw 1.22.7 includes\filebackend folder mw 1.23.0 is working well. I tried a compare but there are a lot of minimal changes on every files

This post was posted by Raid~mediawikiwiki, but signed as Raid.

disable_functions: popen,...,proc_open

That is the problem. MediaWiki actually needs these functions. Does it work, when these functions are enabled?

ok, but what about POPEN function is even on MW 1.22.7 on FSFileBackend.php?

$pipes = array(); foreach ( $fileOpHandles as $index => $fileOpHandle ) { $pipes[$index] = popen( "{$fileOpHandle->cmd} 2>&1", 'r' ); }

EDIT: tested on WAMP 2.4 php 5.4.12 with disable_functions: popen,proc_open

mw 1.23.0 upload system isn't working, mw 1.22.7 is working

popen function is on both versions of mw.

This post was posted by Raid~mediawikiwiki, but signed as Raid.

I haven't looked into it for now, but i'm sure, that popen in 1.27 isn't used for every function, e.g. in file upload required functions. Maybe you never used the functions which uses popen, so you never had problems before. But with the change to MW 1.23 MW require popen for file upload. I'm not sure, if we want to change this, the standard configuration of php.ini has no disabled functions: http://www.php.net//manual/de/ini.core.php

Thx for your support, I hope to have a legacy way for upload module as for php safe_mode on

This post was posted by Raid~mediawikiwiki, but signed as Raid.

The solution for safe_mode = On is to set safe_mode to Off. ;-)

Safe mode was a bad idea, also the PHP developers themselves noticed that and recommend to disable it. Safe mode is deprecated since PHP 5.3 and has finally been removed in 5.4. You should not rely on it. MediaWiki is not made to work with safe_mode = On. If you get it working somehow, be happy, but you should not expect MediaWiki to be changed in a way so that things, which are now broken because of safe_mde being On, will start working in the future. That won't happen.

Tracked in bug 66467

This problem is also in Mediawiki 1.24 alpha and Mediawiki 1.24 wmf

Sure, 1.24 based on 1.23 and will be devloped and not restored ;)

Ok. But could someone fix it so that people can use Mediawiki on a shared host please.

MediaWiki does work on shared hosts so the statement is incorrect. MediaWiki does not work on your shared host because your provider blocks random stuff.

hi but how comes it worked in mediawiki 1.22.

I've setup MediaWiki 1.22.7 with uploads.

I've done an upload, and it works.

Then, I've added system,popen,dl,passthru,proc_open,shell_exec to disabled_functions and now I get this error: Error creating thumbnail: Unable to run external programs, passthru() is disabled.

So it didn't work before. If that's not true, please share how did you configured file uploads.

In principle this does not matter :). The developer team for file management decide to use another basic function to handle the file upload. That some shared host provider think, they can do good things with disable_functions, then we can't, in my opinion, do much things. popen() is explained on official php manual (http://www.php.net/manual/de/function.popen.php) without any warning to deactivate or don't use this function. So it is a basic function enabled by default :)

Please ask your host provider, why they deactivate these functions :)

> In principle this does not matter

Agreed. I mean: What should the MediaWiki developers do? Never change the code? That's not possible. With every change comes the "risk" of introducing a usage of a function, which had not been used before. It is just not feasible to try to prohibit this.

but something was change between mediawiki 1.22 and 1.23.

Yeah, that's totally right, look here: https://github.com/wikimedia/mediawiki-core/blob/REL1_23/includes/filebackend/FSFileBackend.php#L702

This line has changed somewhere between release of 1.22.7 and 1.23 and now uses popen :)

Looking at https://git.wikimedia.org/blame/mediawiki%2Fcore.git/HEAD/includes%2Ffilebackend%2FFSFileBackend.php instead (canonical place) you can see that this line and the call to popen() was added in May 2012 and has not changed since then, so some other code likely has changed which (now?) calls this function.

if we look where mediawiki 1.23 alpha was first release and look between when it was release and now and see what was added.

Which 1.23alpha are you referring to?

I don't know, what you want :) The php fatal error says, where the line we looking for is. I have linked the actual git version of this line and we see, that there is used popen(), in REL1_22 this isn't used. I think we don't need to investigate this more, or? :)

The popen you talk about was introduced early in 2012 -- over a year before 1.22 was released in late 2013.

As others have pointed out, there must be something else causing uploads not to work.

Something was added to that file to cause the issue within the year.

1.23 alpha when it was first released in October but if we look at the 1.23 wmf we may be able to test them and see which one introduced the issue.

Hi I pinpointed when the problem started I tried mediawiki 1.23 wmf 9 and it works with popen disabled and between wmf 10 and 14 is where the problem started. I still need to try the other version but that's when the problem started.

Hi the problem started in mediawiki 1.23 wmf 12.

Thanks for tracking this down

Thanks! With my limited knowledge in File Management i can't find any differences between the function calls in upload process, maybe someone other can better say anything :) -> added backtrace to bug.

Hi my host says popen is disabled for security reasons. please fix it.

If you used MediaWiki with them before, then disabling popen is probably something they did recently.

It sounds like if you want to continue to use MediaWiki with uploaded files, you need to find a new host.

This should have been mentioned in Manual:Installation requirements, but it isn't mentioned.

That's really something to be mentioned, because people may buy a specific hosting plan without being aware that it doesn't just need PHP, but also several PHP functions that hosting services usually consider "restricted" need to be available!

Agreed: added.

Hi I found the source of the problem and where the error originates the file is called FileOpBatch.php and is in include/filebackend/FileOpBatch.php https://git.wikimedia.org/blob/mediawiki%2Fcore.git/36174b76ea9ff6c2e75361865f21b75f212285fa/includes%2Ffilebackend%2FFileOpBatch.php

Hmm, i don't know enugh about file handling, but for me, the function calls seems to be the same? Can you clarify, what you mean with "and where the error originates", the file has 200 lines ;)

Hi I found the code causing the error this is the code in 1.23 causing error

			foreach ( $performOpsBatch as $i => $fileOp ) {
				if ( !isset( $status->success[$i] ) ) { // didn't already fail in precheck()
					$subStatus = $fileOp->attemptAsync();
					if ( $subStatus->value instanceof FileBackendStoreOpHandle ) {
						$opHandles[$i] = $subStatus->value; // deferred
					} else {
						$statuses[$i] = $subStatus; // done already
					}
				}
			}
			// Try to do all the operations concurrently...
			$statuses = $statuses + $backend->executeOpHandlesInternal( $opHandles );
			// Marshall and merge all the responses (blocking)...
			foreach ( $performOpsBatch as $i => $fileOp ) {
				if ( !isset( $status->success[$i] ) ) { // didn't already fail in precheck()
					$subStatus = $statuses[$i];
					$status->merge( $subStatus );
					if ( $subStatus->isOK() ) {
						$status->success[$i] = true;
						++$status->successCount;
					} else {
						$status->success[$i] = false;
						++$status->failCount;
						$aborted = true; // set abort flag; we can't continue
					}
				}
			}
		}
	}
}

and this is the code for 1.22 that works correctly

			foreach ( $performOpsBatch as $i => $fileOp ) {
				if ( !$fileOp->failed() ) { // failed => already has Status
					// If the batch is just one operation, it's faster to avoid
					// pipelining as that can involve creating new TCP connections.
					$subStatus = ( count( $performOpsBatch ) > 1 )
						? $fileOp->attemptAsync()
						: $fileOp->attempt();
					if ( $subStatus->value instanceof FileBackendStoreOpHandle ) {
						$opHandles[$i] = $subStatus->value; // deferred
					} else {
						$statuses[$i] = $subStatus; // done already
					}
				}
			}
			// Try to do all the operations concurrently...
			$statuses = $statuses + $backend->executeOpHandlesInternal( $opHandles );
			// Marshall and merge all the responses (blocking)...
			foreach ( $performOpsBatch as $i => $fileOp ) {
				if ( !$fileOp->failed() ) { // failed => already has Status
					$subStatus = $statuses[$i];
					$status->merge( $subStatus );
					if ( $subStatus->isOK() ) {
						$status->success[$i] = true;
						++$status->successCount;
					} else {
						$status->success[$i] = false;
						++$status->failCount;
						$aborted = true; // set abort flag; we can't continue
					}
				}
			}
		}
		return $status;
	}
}

I am generally opposed to adding needed PHP functions to Manual:Installation requirements. Such a list is not helpful at all. How many functions do you want to add? Hosters can disable any function - do you want to start with stuff like str_replace? That might be disabled as well. We don't know for which strange "reasons" hosters arbitrarily disable stuff - and who should maintain that list? Basically you would have to go through every Gerrit changeset to see, if there might not be a new function, which now is needed as well. Noone will do that - it does not make sense and is not feasable to ad a list of all the functions MediaWiki uses.

Hi I uploaded a change here https://gerrit.wikimedia.org/r/#/c/140383/ which temporarily fixes it.

I have also added the change to branch REL1_23 https://gerrit.wikimedia.org/r/#/c/140387/

88.130: No, there's no need to mention a list of PHP functions, but it will be incredibly helpful to mention that in some circumstances (like file upload) it requires some of them that are usually disabled in the vast majority of free or cheap webhosting plans, basically those that provide the ability to execute external programs.

VipsScaler when they remove the extension issue has been resolved.

This is the fix: https://gerrit.wikimedia.org/r/#/c/140406/2/includes/filebackend/FileOpBatch.php

I just report. Tested, it is working.

Thx for the fix guys.

EDIT: testing version MediaWiki 1.23.0 PHP 5.4.12 (apache2handler) MySQL 5.6.12-log

my phpinfo(): http://pastebin.com/zYedTqa6

EDIT2:

mw 1.23.1 upload module is fully working with popen disabled.

thx for the fix guys

This post was posted by Raid~mediawikiwiki, but signed as Raid.

Hello all! Like Andre K. (thanks :)) said on bugtracker: all patches are merged into the right places, so the master and REL1_23 are fixed. Can you please update your MediaWiki to the newest REL1_23 GIT Version and try uploads? Thanks :)

Alternatively, download the point release of 1.23 this Thursday.