Quantcast

Database Redundancy and Backup and Restore Procedures

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Database Redundancy and Backup and Restore Procedures

Jeremy Lemaire
I am attempting to create a redundant lucidDb server by automating full and differential backup/restores.  Following the documented backup/restore procedures found here...

http://pub.eigenbase.org/wiki/LucidDbSysRoot_RESTORE_DATABASE

...a few questions have arisen:
  1.  When doing a restore it seems important to make sure that no SQL statements are run between the FULL and DIFFERENTIAL restore.  Is this important just for statements that modify the database ( i.e. insert, upsert etc) or does this also hold true for query only statements like a select?
  2. To ensure that no statements are running it is recommended that the sqllineEngine be used instead of the  lucidDbServer-sqllineClient combination.  This seems to imply that before a restore is run that the lucidDbServer must be shutdown however, there does not seem to be a way to do this shutdown in a programmatic manner from a remote server.  Are there any tricks to automating a restore from a remote server or is this generally considered a manual process (i.e. hot backup, cold restore ).  
  3. Is the backup/restore procedure the best way to create/maintain a redundant LucidDb Server?  

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

Zelaine Fong
1.  The restriction only applies to DDL and DML statements, not selects.
2.  The recommendation to use sqllineEngine was due to the existence of bug LDB-190.  Since that's been fixed, you shouldn't have to use sqllineEngine.

-- Zelaine


From: Jeremy Lemaire <[hidden email]>
To: [hidden email]
Sent: Tue, June 8, 2010 11:45:20 AM
Subject: [luciddb-users] Database Redundancy and Backup and Restore Procedures

I am attempting to create a redundant lucidDb server by automating full and differential backup/restores.  Following the documented backup/restore procedures found here...

http://pub.eigenbase.org/wiki/LucidDbSysRoot_RESTORE_DATABASE

...a few questions have arisen:
  1.  When doing a restore it seems important to make sure that no SQL statements are run between the FULL and DIFFERENTIAL restore.  Is this important just for statements that modify the database ( i.e. insert, upsert etc) or does this also hold true for query only statements like a select?
  2. To ensure that no statements are running it is recommended that the sqllineEngine be used instead of the  lucidDbServer-sqllineClient combination.  This seems to imply that before a restore is run that the lucidDbServer must be shutdown however, there does not seem to be a way to do this shutdown in a programmatic manner from a remote server.  Are there any tricks to automating a restore from a remote server or is this generally considered a manual process (i.e. hot backup, cold restore ).  
  3. Is the backup/restore procedure the best way to create/maintain a redundant LucidDb Server?  

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

ngoodman
<base href="x-msg://311/">Thanks Zelaine!  

Also, regarding the shutdown.  JVS and I have been working on the a more daemon friendly version of the LucidDB server (shutdown hooks that will accept OS level signals, etc).  Part of this also has a "shutdown with delay" system proc.  However, it's really fresh code and has just been checked in (few days back) and will make it into the next release, but isn't part of 0.9.2.

I think we've probably nailed the requirements, but any feedback/testing on it is appreciated.

Nick

On Jun 8, 2010, at 12:57 PM, Zelaine Fong wrote:

1.  The restriction only applies to DDL and DML statements, not selects.
2.  The recommendation to use sqllineEngine was due to the existence of bug LDB-190.  Since that's been fixed, you shouldn't have to use sqllineEngine.

-- Zelaine


From: Jeremy Lemaire <[hidden email]>
To: [hidden email]
Sent: Tue, June 8, 2010 11:45:20 AM
Subject: [luciddb-users] Database Redundancy and Backup and Restore Procedures

I am attempting to create a redundant lucidDb server by automating full and differential backup/restores.  Following the documented backup/restore procedures found here...

http://pub.eigenbase.org/wiki/LucidDbSysRoot_RESTORE_DATABASE

...a few questions have arisen:
  1.  When doing a restore it seems important to make sure that no SQL statements are run between the FULL and DIFFERENTIAL restore.  Is this important just for statements that modify the database ( i.e. insert, upsert etc) or does this also hold true for query only statements like a select?
  2. To ensure that no statements are running it is recommended that the sqllineEngine be used instead of the  lucidDbServer-sqllineClient combination.  This seems to imply that before a restore is run that the lucidDbServer must be shutdown however, there does not seem to be a way to do this shutdown in a programmatic manner from a remote server.  Are there any tricks to automating a restore from a remote server or is this generally considered a manual process (i.e. hot backup, cold restore ).  
  3. Is the backup/restore procedure the best way to create/maintain a redundant LucidDb Server?  
------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate 
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
lucky parental unit.  See the prize list and enter to win: 
http://p.sf.net/sfu/thinkgeek-promo_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users


------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

John Sichi
Administrator
In reply to this post by Jeremy Lemaire
Jeremy Lemaire wrote:
>    3. Is the backup/restore procedure the best way to create/maintain a
>       redundant LucidDb Server?  

Yes, if the small downtime for the restore on the slave is acceptable to
you.  If you use a pair of slaves and a load balancer, you can avoid
this too, and worry less about glitches during the restore.

If the downtime is not acceptable, then you're better off coding up some
sort of procedure which periodically moves rows at the logical
level--more work, and less efficient.

Nick and I had some discussions about hot restore in

http://issues.eigenbase.org/browse/LDB-207

but nothing more than talk so far.

JVS

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

Zelaine Fong
In reply to this post by Jeremy Lemaire
Oops, correction to my answer #1 below.  The restriction applies to *all* statements, not just DDL/DML.  Sorry about that.

-- Zelaine


From: Zelaine Fong <[hidden email]>
To: Mailing list for users of LucidDB <[hidden email]>
Sent: Tue, June 8, 2010 12:57:24 PM
Subject: Re: [luciddb-users] Database Redundancy and Backup and Restore Procedures

1.  The restriction only applies to DDL and DML statements, not selects.
2.  The recommendation to use sqllineEngine was due to the existence of bug LDB-190.  Since that's been fixed, you shouldn't have to use sqllineEngine.

-- Zelaine


From: Jeremy Lemaire <[hidden email]>
To: [hidden email]
Sent: Tue, June 8, 2010 11:45:20 AM
Subject: [luciddb-users] Database Redundancy and Backup and Restore Procedures

I am attempting to create a redundant lucidDb server by automating full and differential backup/restores.  Following the documented backup/restore procedures found here...

http://pub.eigenbase.org/wiki/LucidDbSysRoot_RESTORE_DATABASE

...a few questions have arisen:
  1.  When doing a restore it seems important to make sure that no SQL statements are run between the FULL and DIFFERENTIAL restore.  Is this important just for statements that modify the database ( i.e. insert, upsert etc) or does this also hold true for query only statements like a select?
  2. To ensure that no statements are running it is recommended that the sqllineEngine be used instead of the  lucidDbServer-sqllineClient combination.  This seems to imply that before a restore is run that the lucidDbServer must be shutdown however, there does not seem to be a way to do this shutdown in a programmatic manner from a remote server.  Are there any tricks to automating a restore from a remote server or is this generally considered a manual process (i.e. hot backup, cold restore ).  
  3. Is the backup/restore procedure the best way to create/maintain a redundant LucidDb Server?  

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

John Sichi
Administrator
Zelaine, is that because we bump up the CSN even for queries?  I tried a
single query in between, and it actually seemed to work as long as I
didn't skip restoring the catalog, but maybe I didn't try the right
scenario?

(Without the catalog restore clearly it won't work.)

JVS

Zelaine Fong wrote:

> Oops, correction to my answer #1 below.  The restriction applies to
> *all* statements, not just DDL/DML.  Sorry about that.
>
> -- Zelaine
>
> ------------------------------------------------------------------------
> *From:* Zelaine Fong <[hidden email]>
> *To:* Mailing list for users of LucidDB
> <[hidden email]>
> *Sent:* Tue, June 8, 2010 12:57:24 PM
> *Subject:* Re: [luciddb-users] Database Redundancy and Backup and
> Restore Procedures
>
> 1.  The restriction only applies to DDL and DML statements, not selects.
> 2.  The recommendation to use sqllineEngine was due to the existence of
> bug LDB-190.  Since that's been fixed, you shouldn't have to use
> sqllineEngine.
>
> -- Zelaine
>
> ------------------------------------------------------------------------
> *From:* Jeremy Lemaire <[hidden email]>
> *To:* [hidden email]
> *Sent:* Tue, June 8, 2010 11:45:20 AM
> *Subject:* [luciddb-users] Database Redundancy and Backup and Restore
> Procedures
>
> I am attempting to create a redundant lucidDb server by automating full
> and differential backup/restores.  Following the documented
> backup/restore procedures found here...
>
>     http://pub.eigenbase.org/wiki/LucidDbSysRoot_RESTORE_DATABASE
>
> ...a few questions have arisen:
>
>    1.  When doing a restore it seems important to make sure that no SQL
>       statements are run between the FULL and DIFFERENTIAL restore.  Is
>       this important just for statements that modify the database ( i.e.
>       insert, upsert etc) or does this also hold true for query
>       only statements like a select?
>    2. To ensure that no statements are running it is recommended that
>       the sqllineEngine be used instead of the
>        lucidDbServer-sqllineClient combination.  This seems to imply
>       that before a restore is run that the lucidDbServer must be
>       shutdown however, there does not seem to be a way to do this
>       shutdown in a programmatic manner from a remote server.  Are there
>       any tricks to automating a restore from a remote server or is this
>       generally considered a manual process (i.e. hot backup, cold
>       restore ).  
>    3. Is the backup/restore procedure the best way to create/maintain a
>       redundant LucidDb Server?  
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users


------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

Zelaine Fong
Yes, since even read queries bump up the csn, I thought that would invalidate the incremental backups.

-- Zelaine


From: John V. Sichi <[hidden email]>
To: Mailing list for users of LucidDB <[hidden email]>
Sent: Tue, June 8, 2010 10:45:35 PM
Subject: Re: [luciddb-users] Database Redundancy and Backup and Restore Procedures

Zelaine, is that because we bump up the CSN even for queries?  I tried a
single query in between, and it actually seemed to work as long as I
didn't skip restoring the catalog, but maybe I didn't try the right
scenario?

(Without the catalog restore clearly it won't work.)

JVS

Zelaine Fong wrote:

> Oops, correction to my answer #1 below.  The restriction applies to
> *all* statements, not just DDL/DML.  Sorry about that.
>
> -- Zelaine
>
> ------------------------------------------------------------------------
> *From:* Zelaine Fong <[hidden email]>
> *To:* Mailing list for users of LucidDB
> <[hidden email]>
> *Sent:* Tue, June 8, 2010 12:57:24 PM
> *Subject:* Re: [luciddb-users] Database Redundancy and Backup and
> Restore Procedures
>
> 1.  The restriction only applies to DDL and DML statements, not selects.
> 2.  The recommendation to use sqllineEngine was due to the existence of
> bug LDB-190.  Since that's been fixed, you shouldn't have to use
> sqllineEngine.
>
> -- Zelaine
>
> ------------------------------------------------------------------------
> *From:* Jeremy Lemaire <[hidden email]>
> *To:* [hidden email]
> *Sent:* Tue, June 8, 2010 11:45:20 AM
> *Subject:* [luciddb-users] Database Redundancy and Backup and Restore
> Procedures
>
> I am attempting to create a redundant lucidDb server by automating full
> and differential backup/restores.  Following the documented
> backup/restore procedures found here...
>
>    http://pub.eigenbase.org/wiki/LucidDbSysRoot_RESTORE_DATABASE

>
> ...a few questions have arisen:
>
>    1.  When doing a restore it seems important to make sure that no SQL
>      statements are run between the FULL and DIFFERENTIAL restore.  Is
>      this important just for statements that modify the database ( i.e.
>      insert, upsert etc) or does this also hold true for query
>      only statements like a select?
>    2. To ensure that no statements are running it is recommended that
>      the sqllineEngine be used instead of the
>        lucidDbServer-sqllineClient combination.  This seems to imply
>      that before a restore is run that the lucidDbServer must be
>      shutdown however, there does not seem to be a way to do this
>      shutdown in a programmatic manner from a remote server.  Are there
>      any tricks to automating a restore from a remote server or is this
>      generally considered a manual process (i.e. hot backup, cold
>      restore ). 
>    3. Is the backup/restore procedure the best way to create/maintain a
>      redundant LucidDb Server? 
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users


------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and RestoreProcedures

Julian Hyde
Just curious; what does 'CSN' stand for? Oracle has a System Change Number (SCN), and your CSN sounds similar.
 
Reminds me of that phenomenon where the French will come up with an acronym which is an anagram of the English one (e.g. SIDA = AIDS, ADN = DNA, OLP = PLO).
 
Julian

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and RestoreProcedures

Zelaine Fong
Commit Sequence Number.  Every transaction has a unique one.

-- Zelaine


From: Julian Hyde <[hidden email]>
To: Mailing list for users of LucidDB <[hidden email]>
Sent: Wed, June 9, 2010 9:30:08 AM
Subject: Re: [luciddb-users] Database Redundancy and Backup and RestoreProcedures

Just curious; what does 'CSN' stand for? Oracle has a System Change Number (SCN), and your CSN sounds similar.
 
Reminds me of that phenomenon where the French will come up with an acronym which is an anagram of the English one (e.g. SIDA = AIDS, ADN = DNA, OLP = PLO).
 
Julian

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

John Sichi
Administrator
In reply to this post by Zelaine Fong
The definitive answer for 1:  it's OK to issue queries, because we treat
the commit for them as a NOP.  Executing a query causes a new CSN to be
generated, but the CSN of last commit does not change.  (It's like an
abandoned changelist number in Perforce.)

JVS

Zelaine Fong wrote:

> Yes, since even read queries bump up the csn, I thought that would
> invalidate the incremental backups.
>
> -- Zelaine
>
> ------------------------------------------------------------------------
> *From:* John V. Sichi <[hidden email]>
> *To:* Mailing list for users of LucidDB
> <[hidden email]>
> *Sent:* Tue, June 8, 2010 10:45:35 PM
> *Subject:* Re: [luciddb-users] Database Redundancy and Backup and
> Restore Procedures
>
> Zelaine, is that because we bump up the CSN even for queries?  I tried a
> single query in between, and it actually seemed to work as long as I
> didn't skip restoring the catalog, but maybe I didn't try the right
> scenario?
>
> (Without the catalog restore clearly it won't work.)
>
> JVS
>
> Zelaine Fong wrote:
>  > Oops, correction to my answer #1 below.  The restriction applies to
>  > *all* statements, not just DDL/DML.  Sorry about that.
>  >
>  > -- Zelaine
>  >
>  > ------------------------------------------------------------------------
>  > *From:* Zelaine Fong <[hidden email]
> <mailto:[hidden email]>>
>  > *To:* Mailing list for users of LucidDB
>  > <[hidden email]
> <mailto:[hidden email]>>
>  > *Sent:* Tue, June 8, 2010 12:57:24 PM
>  > *Subject:* Re: [luciddb-users] Database Redundancy and Backup and
>  > Restore Procedures
>  >
>  > 1.  The restriction only applies to DDL and DML statements, not selects.
>  > 2.  The recommendation to use sqllineEngine was due to the existence of
>  > bug LDB-190.  Since that's been fixed, you shouldn't have to use
>  > sqllineEngine.
>  >
>  > -- Zelaine
>  >
>  > ------------------------------------------------------------------------
>  > *From:* Jeremy Lemaire <[hidden email]
> <mailto:[hidden email]>>
>  > *To:* [hidden email]
> <mailto:[hidden email]>
>  > *Sent:* Tue, June 8, 2010 11:45:20 AM
>  > *Subject:* [luciddb-users] Database Redundancy and Backup and Restore
>  > Procedures
>  >
>  > I am attempting to create a redundant lucidDb server by automating full
>  > and differential backup/restores.  Following the documented
>  > backup/restore procedures found here...
>  >
>>    http://pub.eigenbase.org/wiki/LucidDbSysRoot_RESTORE_DATABASE
>  >
>  > ...a few questions have arisen:
>  >
>  >    1.  When doing a restore it seems important to make sure that no SQL
>  >      statements are run between the FULL and DIFFERENTIAL restore.  Is
>  >      this important just for statements that modify the database ( i.e.
>  >      insert, upsert etc) or does this also hold true for query
>  >      only statements like a select?
>  >    2. To ensure that no statements are running it is recommended that
>  >      the sqllineEngine be used instead of the
>  >        lucidDbServer-sqllineClient combination.  This seems to imply
>  >      that before a restore is run that the lucidDbServer must be
>  >      shutdown however, there does not seem to be a way to do this
>  >      shutdown in a programmatic manner from a remote server.  Are there
>  >      any tricks to automating a restore from a remote server or is this
>  >      generally considered a manual process (i.e. hot backup, cold
>  >      restore ).
>  >    3. Is the backup/restore procedure the best way to create/maintain a
>  >      redundant LucidDb Server?
>  >
>  >
>  > ------------------------------------------------------------------------
>  >
>  >
> ------------------------------------------------------------------------------
>  > ThinkGeek and WIRED's GeekDad team up for the Ultimate
>  > GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
>  > lucky parental unit.  See the prize list and enter to win:
>>  http://p.sf.net/sfu/thinkgeek-promo
>  >
>  >
>  > ------------------------------------------------------------------------
>  >
>  > _______________________________________________
>  > luciddb-users mailing list
>  > [hidden email]
> <mailto:[hidden email]>
>  > https://lists.sourceforge.net/lists/listinfo/luciddb-users
>
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> <mailto:[hidden email]>
> https://lists.sourceforge.net/lists/listinfo/luciddb-users
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users


------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

Jeremy Lemaire
I am no longer able to do a DIFFERENTIAL backup in a timely manner.  The FULL backup, although it took nearly 12 hours to complete, seemed to succeed.  I have given the DIFFERENTIAL over 12 hours and still does not finish and I am unable to kill the statement or session.  The kill_* just hangs.  Generally the DIFFERENTIAL is quicker so it seems like something is wrong.

To give a little background on the problem, I was doing daily DIFFERENTIAL backups which seemed to be taking longer and longer as the db.dat grew so my solution was to do a fresh FULL backup to minimize the DIFFERENTIAL.  It was the subsequent DIFFERENTIALS after this new FULL that I am haveing problems with.

The data base is currently about 1T in size and the FULL backup is about 271G compressed.  The DIFFERENTIAL backup is hanging at around 9.7G and there is no backup.properties file created.  According to ps it appears to be stuck gzipping the FennelDataDump.dat.gz file when the hang occurs.

Currently I am running with no backup of the data base and no way to dump DDL to create a backup from raw data.  Do you have any suggestions?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

John Sichi
Administrator
Is there any CPU usage on the gzip or luciddb processes?

Have you tried shutting your server down and restarting it?

JVS

haawker wrote:

> I am no longer able to do a DIFFERENTIAL backup in a timely manner.  The FULL
> backup, although it took nearly 12 hours to complete, seemed to succeed.  I
> have given the DIFFERENTIAL over 12 hours and still does not finish and I am
> unable to kill the statement or session.  The kill_* just hangs.  Generally
> the DIFFERENTIAL is quicker so it seems like something is wrong.
>
> To give a little background on the problem, I was doing daily DIFFERENTIAL
> backups which seemed to be taking longer and longer as the db.dat grew so my
> solution was to do a fresh FULL backup to minimize the DIFFERENTIAL.  It was
> the subsequent DIFFERENTIALS after this new FULL that I am haveing problems
> with.
>
> The data base is currently about 1T in size and the FULL backup is about
> 271G compressed.  The DIFFERENTIAL backup is hanging at around 9.7G and
> there is no backup.properties file created.  According to ps it appears to
> be stuck gzipping the FennelDataDump.dat.gz file when the hang occurs.
>
> Currently I am running with no backup of the data base and no way to dump
> DDL to create a backup from raw data.  Do you have any suggestions?


------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

FarragoDebugServer errors

Hao Loi

I set up a Netbeans project for Farrago and I have resolved all the errors.
But when I tried to debug the FarragoDebugServer.java, I got the following
errors.  Could anyone help me to resolve the following problems:

debug:
Exception in thread "main" java.util.MissingResourceException: Can't find
bundle for base name net.sf.farrago.resource.FarragoResource, locale en_US
        at
java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1521)
        at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1260)
        at java.util.ResourceBundle.getBundle(ResourceBundle.java:787)
        at
net.sf.farrago.resource.FarragoResource.instance(FarragoResource.java:29)
        at
net.sf.farrago.server.FarragoAbstractServer.start(FarragoAbstractServer.java:220)
        at
net.sf.farrago.server.FarragoVjdbcServer.main(FarragoVjdbcServer.java:79)
        at net.sf.farrago.test.FarragoDebugServer.main(FarragoDebugServer.java:50)
Caused by: java.io.IOException: could not open properties file for class
net.sf.farrago.resource.FarragoResource
        at org.eigenbase.resgen.ShadowResourceBundle.<init>(Unknown Source)
        at net.sf.farrago.resource.FarragoResource.<init>(FarragoResource.java:20)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2389)
        at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1406)
        at java.util.ResourceBundle.findBundle(ResourceBundle.java:1365)
        at java.util.ResourceBundle.findBundle(ResourceBundle.java:1292)
        at java.util.ResourceBundle.findBundle(ResourceBundle.java:1292)
        at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1234)
        ... 5 more
Java Result: 1
BUILD SUCCESSFUL (total time: 12 seconds)

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

Jeremy Lemaire
In reply to this post by John Sichi
I did not get a chance to check CPU usage because the machine eventually had a drive failure.    

Once the redundant node came online I was able to successfully do a FULL (12hrs) and DIFFERENTIAL (20 mins) backup in a reasonable amount of time.

Therefore I am assuming this bad drive was at least partially contributing to the backup issues I was seeing.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

running error

Hao Loi
In reply to this post by John Sichi

I almost have the FarragoDebugServer running on my Netbeans excpet for the
following errors.  Could someone please help me to figure out what is wrong?

Server personality:  Eigenbase Data Management System
Loading database...
Aug 19, 2010 4:16:56 PM net.sf.farrago.db.FarragoDbSingleton pinReference
INFO: connect
Aug 19, 2010 4:18:24 PM org.eigenbase.util.EigenbaseException <init>
SEVERE: org.eigenbase.util.EigenbaseException: Installation error: required
Java property java.util.logging.config.file not set
Aug 19, 2010 4:18:24 PM net.sf.farrago.db.FarragoDatabase warnOnClose
WARNING: Caught java.lang.NullPointerException during database shutdown:null
Aug 19, 2010 4:18:24 PM org.eigenbase.util.EigenbaseException <init>
SEVERE: org.eigenbase.util.EigenbaseException: Failed to load database
Exception in thread "main" org.eigenbase.util.EigenbaseException: Failed to
load database
        at
net.sf.farrago.resource.FarragoResource$_Def1.ex(FarragoResource.java:2021)
        at net.sf.farrago.db.FarragoDatabase.<init>(FarragoDatabase.java:292)
        at
net.sf.farrago.db.FarragoDbSingleton.pinReference(FarragoDbSingleton.java:100)
        at
net.sf.farrago.server.FarragoAbstractServer.start(FarragoAbstractServer.java:232)
        at
net.sf.farrago.server.FarragoVjdbcServer.main(FarragoVjdbcServer.java:79)
        at net.sf.farrago.test.FarragoDebugServer.main(FarragoDebugServer.java:50)
Caused by: org.eigenbase.util.EigenbaseException: Installation error: required
Java property java.util.logging.config.file not set
        at
net.sf.farrago.resource.FarragoResource$_Def0.ex(FarragoResource.java:1998)
        at net.sf.farrago.db.FarragoDatabase.<init>(FarragoDatabase.java:145)
        ... 4 more
Java Result: 1
BUILD SUCCESSFUL (total time: 1 minute 44 seconds)


------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: running error

John Sichi
Administrator
You should study farrago/defineFarragoRuntime.sh to see how it is
setting up the environment, classpath, and JVM; you'll need the
process launched by Netbeans to match all of that.

JVS

On Thu, Aug 19, 2010 at 1:18 PM, Hao Loi <[hidden email]> wrote:

>
> I almost have the FarragoDebugServer running on my Netbeans excpet for the
> following errors.  Could someone please help me to figure out what is wrong?
>
> Server personality:  Eigenbase Data Management System
> Loading database...
> Aug 19, 2010 4:16:56 PM net.sf.farrago.db.FarragoDbSingleton pinReference
> INFO: connect
> Aug 19, 2010 4:18:24 PM org.eigenbase.util.EigenbaseException <init>
> SEVERE: org.eigenbase.util.EigenbaseException: Installation error: required
> Java property java.util.logging.config.file not set
> Aug 19, 2010 4:18:24 PM net.sf.farrago.db.FarragoDatabase warnOnClose
> WARNING: Caught java.lang.NullPointerException during database shutdown:null
> Aug 19, 2010 4:18:24 PM org.eigenbase.util.EigenbaseException <init>
> SEVERE: org.eigenbase.util.EigenbaseException: Failed to load database
> Exception in thread "main" org.eigenbase.util.EigenbaseException: Failed to
> load database
>        at
> net.sf.farrago.resource.FarragoResource$_Def1.ex(FarragoResource.java:2021)
>        at net.sf.farrago.db.FarragoDatabase.<init>(FarragoDatabase.java:292)
>        at
> net.sf.farrago.db.FarragoDbSingleton.pinReference(FarragoDbSingleton.java:100)
>        at
> net.sf.farrago.server.FarragoAbstractServer.start(FarragoAbstractServer.java:232)
>        at
> net.sf.farrago.server.FarragoVjdbcServer.main(FarragoVjdbcServer.java:79)
>        at net.sf.farrago.test.FarragoDebugServer.main(FarragoDebugServer.java:50)
> Caused by: org.eigenbase.util.EigenbaseException: Installation error: required
> Java property java.util.logging.config.file not set
>        at
> net.sf.farrago.resource.FarragoResource$_Def0.ex(FarragoResource.java:1998)
>        at net.sf.farrago.db.FarragoDatabase.<init>(FarragoDatabase.java:145)
>        ... 4 more
> Java Result: 1
> BUILD SUCCESSFUL (total time: 1 minute 44 seconds)
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Database Redundancy and Backup and Restore Procedures

John Sichi
Administrator
In reply to this post by Jeremy Lemaire
Thanks for the update Jeremy...from what you reported about
performance progressively degrading, that does sound very much like a
storage hardware problem.

JVS

On Thu, Aug 19, 2010 at 9:02 AM, Jeremy Lemaire <[hidden email]> wrote:

>
> I did not get a chance to check CPU usage because the machine eventually had
> a drive failure.
>
> Once the redundant node came online I was able to successfully do a FULL
> (12hrs) and DIFFERENTIAL (20 mins) backup in a reasonable amount of time.
>
> Therefore I am assuming this bad drive was at least partially contributing
> to the backup issues I was seeing.
>
> --
> View this message in context: http://luciddb-users.1374590.n2.nabble.com/Database-Redundancy-and-Backup-and-Restore-Procedures-tp5154973p5440962.html
> Sent from the luciddb-users mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: running error

Hao Loi
In reply to this post by John Sichi
I have my Eclipse project set up properly now and I don't have anymore errors.
 Except when I start the debugger, it never loads the database successfully.
It stops at the following:

In the debugger:
Server personality:  Farrago
Loading database...

and in class FarragoDbSingleton it stops at


    public static synchronized FarragoDatabase pinReference(
        FarragoSessionFactory sessionFactory)
    {
        tracer.info("connect");

        ++nReferences;
        if (nReferences == 1) {
            assert (instance == null);
            boolean success = false;
            try {
                FarragoDatabase newDb =
                    new FarragoDatabase(sessionFactory, false);
                assert (newDb == instance);
                success = true;
            } finally {
                if (!success) {
                    nReferences = 0;
stop here----->     instance = null;
                }
            }
        }
        return instance;
    }



I starts the sqllineClient and it can't connect to the server.


Please help.


------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: running error

Hao Loi

You are right about the $LD_LIBRARY_PATH.  After I put in a full path when
running fennelenv.sh, it's working now.  Please ignore my previous post.
Thanks again for your help.

Hao

------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Loading...