Converting from Derby to MySQL

This message was edited 3 times. Last update was at 25/06/2009 09:47:19

If your instance of Mango has an installation of MySQL "nearby", you will likely be able to achieve much better overall performance by converting databases.

The following is a basic set of instructions for converting from Derby to MySQL. Since 1.6.2, Mango shipped with a conversion utility that automatically will copy your data from Derby to a fresh MySQL schema (and theoretically in the other direction as well, although i'm not personally aware of anyone having done it). Note that one user reported issues when using this utility on a non-standard host, but that attempts have otherwise been successful.

If this procedure does not work for you, please report your errors so that others can benefit from the experience.

1) Ensure that your existing Derby instance is up to date with your Mango software. That is, do not download a new version of the software and attempt a conversion without giving Mango a chance to upgrade your existing database. Conversions will only work when source and target are exactly the same version.

2) Create a new schema in your MySQL instance and provide a user account with full permissions to it. DO NOT create the Mango tables. Mango likes to do this itself.

3) Edit the WEB-INF/classes/env.properties file. You will have existing "db." entries in there that point to the Derby database. Prepend these entries with "convert." They will then look something like this:

Note that you may only have the "url" entry. This is fine.

4) Add entries that point to your MySQL instance. The following are examples. Modify them to suit your environment.

Check the MySQL documentation for more information on connection URLs if you need help. The last two entries are for connection pooling configuration and are optional. Note that the ordering of these properties in the file is inconsequential.

5) Save the changes.

6) Important: check your WEB-INF/lib directory for old versions of the MySQL JDBC library. The correct version to have in there is mysql-connector-java-5.1.7-bin.jar. Remove any other versions, such as mysql-connector-java-3.1.8-bin.jar.

7) Restart Mango.

8) Watch the console messages. You will be given information on the steps and progress of the conversion as it occurs. Depending on the size of your Derby database and the power of your hardware this process could take seconds or hours.

BTW, if you have any problems during conversion (e.g. a power failure, and error, etc), you can always restart the process by dropping all of the tables that Mango created in your MySQL schema and restarting Mango.

Dear sir,

I try change databse from Derby to Mysql error message:

INFO 2010-01-07 15:26:17,921 (org.springframework.web.context.ContextLoader.initWebApplicationContext:189) - Root WebApplicationContext: initialization started
INFO 2010-01-07 15:26:17,968 (org.springframework.context.support.AbstractApplicationContext.prepareRefresh:412) - Refreshing org.springframework.web.context.support.XmlWebApplicationContext@e0a386: display name [Root WebApplicationContext]; startup date [Thu Jan 07 15:26:17 ICT 2010]; root of context hierarchy
INFO 2010-01-07 15:26:18,046 (org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions:323) - Loading XML bean definitions from ServletContext resource [/WEB-INF/applicationContext.xml]
INFO 2010-01-07 15:26:18,093 (org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory:427) - Bean factory for application context [org.springframework.web.context.support.XmlWebApplicationContext@e0a386]: org.springframework.beans.factory.support.DefaultListableBeanFactory@609959
INFO 2010-01-07 15:26:18,203 (org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons:414) - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@609959: defining beans [CompoundEventsDwr,CustomViewDwr,DataPointDetailsDwr,DataPointEditDwr,DataSourceEditDwr,DataSourceListDwr,EmportDwr,EventHandlersDwr,EventsDwr,MailingListsDwr,MiscDwr,PointHierarchyDwr,PointLinksDwr,PublisherEditDwr,PublisherListDwr,ReportsDwr,ScheduledEventsDwr,SystemSettingsDwr,UsersDwr,ViewDwr,WatchListDwr,localeResolver,messageSource]; root of factory hierarchy
INFO 2010-01-07 15:26:18,343 (org.springframework.web.context.ContextLoader.initWebApplicationContext:209) - Root WebApplicationContext: initialization completed in 406 ms
INFO 2010-01-07 15:26:18,343 (com.serotonin.mango.MangoContextListener.contextInitialized:73) - Mango context starting
INFO 2010-01-07 15:26:18,656 (com.serotonin.mango.db.BasePooledAccess.initializeImpl:54) - Initializing pooled connection manager
WARN 2010-01-07 15:26:19,046 (com.serotonin.mango.db.upgrade.DBUpgrade.checkUpgrade:81) - Upgrading instance from 0.7.0 to 0.8.0
INFO 2010-01-07 15:26:19,062 (org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions:323) - Loading XML bean definitions from class path resource [org/springframework/jdbc/support/sql-error-codes.xml]
INFO 2010-01-07 15:26:19,109 (org.springframework.jdbc.support.SQLErrorCodesFactory.<init>:12

- SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase]
INFO 2010-01-07 15:26:19,125 (com.serotonin.mango.MangoContextListener.contextDestroyed:101) - Mango context terminating
INFO 2010-01-07 15:26:19,125 (com.serotonin.mango.db.BasePooledAccess.terminate:123) - Stopping database
INFO 2010-01-07 15:26:19,125 (com.serotonin.mango.MangoContextListener.contextDestroyed:121) - Mango context terminated
INFO 2010-01-07 15:26:19,125 (org.springframework.context.support.AbstractApplicationContext.doClose:816) - Closing org.springframework.web.context.support.XmlWebApplicationContext@e0a386: display name [Root WebApplicationContext]; startup date [Thu Jan 07 15:26:17 ICT 2010]; root of context hierarchy
INFO 2010-01-07 15:26:19,125 (org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons:392) - Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@609959: defining beans [CompoundEventsDwr,CustomViewDwr,DataPointDetailsDwr,DataPointEditDwr,DataSourceEditDwr,DataSourceListDwr,EmportDwr,EventHandlersDwr,EventsDwr,MailingListsDwr,MiscDwr,PointHierarchyDwr,PointLinksDwr,PublisherEditDwr,PublisherListDwr,ReportsDwr,ScheduledEventsDwr,SystemSettingsDwr,UsersDwr,ViewDwr,WatchListDwr,localeResolver,messageSource]; root of factory hierarchy

How to solve is problem?

Did you manually create the mysql tables? If so, drop them all and start again. You only need to create the database. Mango need to create the tables itself.

I followed the spteps but after restarting Mango, the tables were created in the MySQL schema but there was no data on them bt the admin/amdin user/password.

Done. I also needed to comment the following lines:

#convert.db.type=
#convert.db.url=
#convert.db.username=
#convert.db.password=

I'm running Mango 1.9.0 and recently converted my derby db to mysql version 5.5.8. I ran into a couple of issues which I was able figure out and fix. Note: these issues may have already been addressed in the newer versions of Mango.

1. There is a new version of the mysql java connector jar available from the mysql website. Download mysql-connector-java-5.1.14.tar.gz, unzip it, and place mysql-connector-java-5.1.14-bin.jar in WEB-INF/lib directory.

2. Several sql scripts and db upgrade source files used a deprecated mysql keyword in 'create table' statements. Search for 'type=InnoDB' and replace with 'engine=InnoDB'.

You should also delete any previous versions of the connector jar file.

Hello, I am having trouble converting to MySQL - I have tried all the above and no success - I am running version 1.12.4. Logs as follows:

un 9, 2011 2:05:30 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext
Jun 9, 2011 2:05:30 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class com.serotonin.mango.MangoContextListener
java.lang.IllegalArgumentException: No enum const class com.serotonin.mango.db.DatabaseAccess$DatabaseType.MYSQL
at java.lang.Enum.valueOf(Enum.java:214)
at com.serotonin.mango.db.DatabaseAccess$DatabaseType.valueOf(DatabaseAccess.java:4

at com.serotonin.mango.db.DatabaseAccess.createDatabaseAccess(DatabaseAccess.java:73)
at com.serotonin.mango.MangoContextListener.databaseInitialize(MangoContextListener.java:253)
at com.serotonin.mango.MangoContextListener.contextInitialized(MangoContextListener.java:94)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3934)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4429)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:987)
at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:909)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:495)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:516)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:28

at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Jun 9, 2011 2:05:30 PM org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext

env.properties

convert.db.type=derby
convert.db.url=~/../../bin/mangoDB
convert.db.username=admin
convert.db.password=admin

db.type=mysql
db.url=jdbc:mysql://localhost/mangodb
db.username=mango
db.password=mango
db.pool.maxActive=10
db.pool.maxIdle=10

Any help much appreciated.

I have also tried with a fresh copy of Mango with just the MySQL settings but no success..

Thanks Gus

I have the same problem that gus mentioned before. Is there a solution? anyone has a suggestion?

Hi I'm having trouble with this conversion. I've performed the steps listed on the first page, but when I go to localhost:8080 I get a HTTP Status 404 Error: The requested resource is not available.

On the Tomcat console, the errors appear to start around here:

SEVERE: Error listenerStart
Jan 24, 2012 9:35:40 AM org.apache.catalina.core.StandardContext start
SEVERE: Context [] startup failed due to previous errors
INFO 2012-01-24 09:35:40,093 (com.serotonin.mango.MangoContextListener.contextD
estroyed:124) - Mango context terminating
INFO 2012-01-24 09:35:40,093 (com.serotonin.mango.db.BasePooledAccess.terminate
:123) - Stopping database

Anyone have any idea why this happens? My Information for the MySQL connection is correct, as when I put in a wrong password on purpose I get different and more errors.

Additionally, does anyone know the user/password login information to log into the derby database? I would like to browse the database with a program I have, but havent been able to make the connection as I havent found where the user/password is kept.

Thanks!

Richard, check the <tomcat>/logs/localhost<date>.log file for error messages.