I’ve created a site in WordPress on our development machine. In the theme we’re using there are numerous widget zones to display text in (sidebar and front page). I’ve used simple Text widgets in all of these zones to put our display information.
When I migrated the site to production, I used the WP-DB-Backup plugin to take a snapshot of the database. I then edited the resulting .sql file to update all of the file paths and URL references to point to our production site.
After creating the database, website, and copying all of the files over to the production site, I run the .sql file from the mysql command prompt to import the data into the new database.
However, when I go to the production site, some of the text shows up and some of it doesn’t. When I look into the widgets section of the site, the text widgets are missing from some of the widget zones. The text widgets aren’t even visible in the “Inactive Widget” zone, they simply aren’t there.
I’ve even tried to repeat the process using the BackWPup plugin, noticing that the SQL syntax is different when it dumps the database out.
Why am I losing text widget data during the import?
1
5 Answers
This is where your problem is:
I then edited the resulting .sql file
to update all of the file paths and
URL references to point to our
production site.
You can’t do that. WordPress stores many options as “serialized data”, which contains both the string content of things and their length. So when you modify the URL and the length changes, then the serialized data is no longer correct, and PHP rejects it.
The longer term problem is that, basically, you’re doing it wrong. If you are setting up a development site that will have its data migrated, then it should have the exact same URL as your production site to begin with. You can manually edit your HOSTS file to give that production domain (like example.com) a different IP address (like 127.0.0.1) and thus the “production” URL will become the development site, for you only. Then you can create your data and links and everything else using that production URL, and when you migrate the data, nothing about it has to be altered.
In the short term, however, don’t use a simple text search/replace on the SQL file. As you have discovered, this breaks things.
And while I hesitate to suggest it, there is a way to alter the WordPress core code to handle these broken serializations. You have to modify the wp-includes/functions.php file, and change the maybe_unserialize() function to this:
function maybe_unserialize( $original ) {
if ( is_serialized( $original ) ) {
$fixed = preg_replace_callback(
'!(?<=^|;)s:(d+)(?=:"(.*?)";(?:}|a:|s:|b:|i:|o:|N;))!s',
'serialize_fix_callback',
$original );
return @unserialize( $fixed );
}
return $original;
}
function serialize_fix_callback($match) { return 's:' . strlen($match[2]); }
This is NOT a viable long term solution. It should only be used to get you up and working right now. In the long run, you need to fix your development process so that you don’t have to do this sort of URL munging to begin with.
11
@Otto excellent answer. Quick question, would modifying a non serialized blob/text table like wp_posts outside of MySql effect any of the serialized data in wp_post_meta or wp_options? I had the same problem with the text widget but I didn’t touch wp_options I only modified wp_posts.
– Chris_OWow, I never realized that is what was happening with the data, but it makes complete sense! Many thanks!
–Another workaround that some people use is to make their development system have a domain name of “example.dev” instead of “example.com”. That way, lengths don’t change for strings when they move them to production. I prefer the HOSTS file method.
– Otto2016 and wordrepss is still saving serialized data in database.
most famous worst codeprize has to look no further.– EjazTHANK YOU!!! Good point and great hack. Generaly I get this hack to return back all data and after that just update existing settings again and when remove this code work perfect.
To deal with this issue I always use WordPress Serialized Search & Replace tool provided here. It works perfectly fine with out any issues. I have been using this for a long time on all my site migration requirements. This really take care of the issues with migrating development database to production.
https://interconnectit.com/products/search-and-replace-for-wordpress-databases/
5
Yes been using this script for years and highly recommend it
– davemacWorked for me most of the time. But this week when I replaced
http://localhost/Me/site_namebyhttp://site.dev(from one local host to another) using v 3.0.0 I did lose my widget and menu positions oddly enough. So perhaps this issue is related to the string lenght as well.– rhandI have been using.. but never faced this situation as of yet. Can you download the older version of this script and try with that again. Try replacing
localhost/Me/site_namewithsite.dev.Url has changed (now https instead http) : interconnectit.com/products/…
– KoryonikGorgeous script. I duplicated a MySQL database from PHPMyAdmin from an old one to a new one -no change of URLs whatsoever-, then went to the folder of the new site where the fresh WP files were (alongside a proper wp-config.php, with the new DB credentials), added the script, and it took care of everything. Serialized data is updated along the normal URLs. Easy and quick! Highly recommended. Important: don’t forget to remove the script after it’s been used as it has access to your DB details!
– Peanuts
Otto’s answer is spot-on. I also discovered this the hard way.
However, I managed to work around this using a cool script at http://spectacu.la/search-and-replace-for-wordpress-databases/
To migrate your wordpress and to a new url / domain name, do the following:
- Take a DB dump (e.g. using phpmyadmin) of the existing wordpress
- Restore the dump as-is, (no need for modifications) to your new location
- Unzip the script from spectacu.la into your wordpress home folder (it’s not a plugin…)
- Run the script on your new site by pointing your browser to it, e.g. http://new-website.url/searchreplacedb.php
- Don’t forget to delete the script from your new wordpress home
4
I know this is kind of old, but where do I supposed to specify the new database name if I restoring the dump as it is? shouldn’t I at least put the new database name in the second step? Thank you for this info
I’m not sure I fully understand your question. Restoring the database can be done with tools like phpmyadmin and you can give it a new name, or use the old name. The script I mentioned simply changes text inside the database after it is already restored.
Hi Yoav, thanks for the answer, I mean, when I export a DB I normally change the database name to the new one and changes the domain links. Said this, on ur step number two you say restore the dump as-is w/o modifications, I just wanted to know if that literally, or I have to change the database name at least. I know it can a be a dummy question, I’m just kind of lost, thanks again for ur answer
I don’t know how you dump your database, but if you use phpmyadmin ‘export’ tool, then it doesn’t matter which database name it is. You can use the export and import it back to any other database. Generally, regarding bullet-point 2, I think that’s ok to change the database name.
The OP was overzealous when doing a search-and-replace on the database export file, and ended up changing occurrences of “wp_” within some of the serialized data. The solution is to be more parsimonious in the search-and-replace by including the backtick within the regular expression, and then manually updating the remaining keys within the database after import.
If you are migrating and changing the prefix, and like a more manual approach, do the following (this only addresses the OPs concerns and does not deal with updating the site URL)
- Backup and move your database export
SQL file to the new environment (my example assumes a filename of backup_YYYY-MM-DD.sql) - Do a mass search-and-replace on the
SQL file to change the table names
to use your new prefix (PRIOR to
importing your SQL file!). One way
to do this would be to use a Perl
one-liner like: perl -p -i.bak -e
“s/`wp_/`myprefix_/g”
backup_YYYY-MM-DD.sql - Import your SQL data into the database
- Update any keys within _options that
contain the prefix hard-coded:
update myprefix_options set
option_name =
concat(‘myprefix_’,substr(option_name,4))
where option_name like ‘wp_%’ - Update any keys within _user_meta
that contain the hard-coded prefix:
update myprefix_usermeta set
meta_key =
concat(‘myprefix_’,substr(meta_key,4))
where meta_key like ‘wp_%’
I used WP Migrate plugin, witch replaces http and folder patches.
I got a single problem when importing, but resolved putting the following lines at the top of the generated sql:
/*!40101 SET @[email protected]@CHARACTER_SET_CLIENT */;
/*!40101 SET @[email protected]@CHARACTER_SET_RESULTS */;
/*!40101 SET @[email protected]@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @[email protected]@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @[email protected]@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @[email protected]@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @[email protected]@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @[email protected]@SQL_NOTES, SQL_NOTES=0 */;
I also tried with the Search And Replace tool (v2.1) replied by @Yoav, but it still breaks my serialized data.
1
Hi Ricardo, Welcome to WordPress Answers! The area you posted in is reserved for answers to the original question. Even though your question is related you should post it as a separate question. You’ll get a much better chance of having it answered that way.
– Chris_O
I’ve been doing some digging along the way, and the only thing I can think of is that the widget information is being stored in the wp_options table, which looks to encode some of its data in a weird way. I haven’t been able to try this with a different theme yet to see if it is theme related.