Re: concurrent updates - in egw the last wins?

Raphael_Alla-2 2008-02-27 10:29:27 UTC #1

In my opinion, ‘the last wins’ can be an issue, but its not so bad as long
as you ensure that the integrity of the data remains. The real issue comes
when there is a serie of database updates, that are happening
simultaneously, and a race condition may lead to the data losing its
integrity.

In Blueerp (a set of modules for egw), this is handled by using sql
transactions.

However, what is not handled is the ‘the last wins’ issue, because in
reality it almost never happens, and that with transactions the data always
stays coherent.

hi devs,
until now i was worried about small lacks of quality in egw1.x, but now i
read in an interview:

The most problems (with egw) occur if there are simultaneous accesses to
the
datasets.
In egw “the last wins”.

until now, i thought, concurrent update is handled by sql - so why does
not
work in egw?
until now, i had no courage to make installation for huge amount of users
with egw,
so i can not check wether concurrent update works - or last wins - as
stated
above.
i will test it soon next weeks - that might be a nuclear bomb for all user
data.

this is a question concerning life or death for users data and infos,
so please all devs give true and reliable infos on this problem for egw.
wolf

ps: such possible lack of integrity might cause big loss of confidence.

View this message in context:
http://www.nabble.com/concurrent-updates---in-egw-the-last-wins-----tp15567236s3741p15567236.html
Sent from the egroupware-developers mailing list archive at Nabble.com.

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

Ralf_Becker_6 2008-02-27 11:07:10 UTC #2

Hi Raphael,

eGW already deals with the last wins situation in the followig apps:

infolog (since 1.2)
calendar (in trunk)
groupdav (1.4)

Though only calendar uses a real counter, the other two use the last
modification date at the moment.

Data integrety or transactions is only a problem, if you have multiple
tables (or updates), as a single sql statement always runs as transaction.

That means it’s currently only critical with the calendar, where we
should add transactions around multiple table/statement updates.
I want to add a real update counter to addressbook and infolog too, as I
need them as etags for GroupDAV and CalDAV.

Ralf

Raphael Alla schrieb:

In my opinion, ‘the last wins’ can be an issue, but its not so bad as
long as you ensure that the integrity of the data remains. The real
issue comes when there is a serie of database updates, that are
happening simultaneously, and a race condition may lead to the data
losing its integrity.

In Blueerp (a set of modules for egw), this is handled by using sql
transactions.

However, what is not handled is the ‘the last wins’ issue, because in
reality it almost never happens, and that with transactions the data
always stays coherent.

2008/2/20, gcx <globalclown@web.de mailto:globalclown@web.de>:
hi devs,
until now i was worried about small lacks of quality in egw1.x, but
now i
read in an interview:

The most problems (with egw) occur if there are simultaneous
accesses to the
datasets.
In egw "the last wins".

until now, i thought, concurrent update is handled by sql - so why
does not
work in egw?
until now, i had no courage to make installation for huge amount of
users
with egw,
so i can not check wether concurrent update works - or last wins -
as stated
above.
i will test it soon next weeks - that might be a nuclear bomb for
all user
data.

this is a question concerning life or death for users data and infos,
so please all devs give true and reliable infos on this problem for egw.
wolf

ps: such possible lack of integrity might cause big loss of confidence.
--
View this message in context:
http://www.nabble.com/concurrent-updates---in-egw-the-last-wins-----tp15567236s3741p15567236.html
Sent from the egroupware-developers mailing list archive at
Nabble.com <http://Nabble.com>.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/egroupware-developers
–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

Raphael_Alla-2 2008-02-27 11:53:24 UTC #3

Hi Ralf,

thanks for this answer. I did not realise that some apps already had a way
to handle the ‘last wins’ situation, I’ll check that.

The other issue, multiple sql updates, seem to occur a lot in my code so I
was more aware of it and took the necessary steps to make sure I used
transactions where needed. At this stage, I believe the issue should not be
addressed on a case by case basis but by allowing egroupware to better
define its data model by:

allowing to define in which context the multiple tables or updates take
place (typically when you have two objects connected by a 1 to 1, 1 to n, or
n to n relationship)
improve/extend the existing so_sql class to handle these situations as
they should

I can propose some code that does the above; and I believe these issues have
been thought about in the design of Tine2. Yet at the same time I wonder if
either using existing libraries or designs from other open source project
may not be a better approach (why reinvent the wheel, but will the wheel of
other projects fit egroupware?)

A generic approach for the issue of ‘the last wins’ should be looked at as
well.

Hi Raphael,

eGW already deals with the last wins situation in the followig apps:

infolog (since 1.2)

calendar (in trunk)

groupdav (1.4)

Though only calendar uses a real counter, the other two use the last
modification date at the moment.

Data integrety or transactions is only a problem, if you have multiple
tables (or updates), as a single sql statement always runs as transaction.

That means it’s currently only critical with the calendar, where we
should add transactions around multiple table/statement updates.
I want to add a real update counter to addressbook and infolog too, as I
need them as etags for GroupDAV and CalDAV.

Ralf

Raphael Alla schrieb:

In my opinion, ‘the last wins’ can be an issue, but its not so bad as
long as you ensure that the integrity of the data remains. The real
issue comes when there is a serie of database updates, that are
happening simultaneously, and a race condition may lead to the data
losing its integrity.

In Blueerp (a set of modules for egw), this is handled by using sql
transactions.

However, what is not handled is the ‘the last wins’ issue, because in
reality it almost never happens, and that with transactions the data
always stays coherent.

2008/2/20, gcx <globalclown@web.de mailto:globalclown@web.de>:
hi devs,
until now i was worried about small lacks of quality in egw1.x, but
now i
read in an interview:

The most problems (with egw) occur if there are simultaneous
accesses to the
datasets.
In egw "the last wins".

until now, i thought, concurrent update is handled by sql - so why
does not
work in egw?
until now, i had no courage to make installation for huge amount of
users
with egw,
so i can not check wether concurrent update works - or last wins -
as stated
above.
i will test it soon next weeks - that might be a nuclear bomb for
all user
data.

this is a question concerning life or death for users data and
infos,
so please all devs give true and reliable infos on this problem for
egw.
wolf

ps: such possible lack of integrity might cause big loss of
confidence.
--
View this message in context:
http://www.nabble.com/concurrent-updates---in-egw-the-last-wins-----tp15567236s3741p15567236.html
Sent from the egroupware-developers mailing list archive at
Nabble.com <http://Nabble.com>.
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/egroupware-developers
–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–

Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

Ralf_Becker_6 2008-02-27 12:01:02 UTC #4

Hi Raphael,

so_sql does not have the problem, as it works unmodified/un-extended
only with one table. It’s the derived classes, which eg. use multiple
so_sql instances to deal with multiple tables, who need to start the
transation.

The best way to deal with the last update win situation is a query like:

UPDATE table SET etag=etag+1, WHERE id=… AND etag=123

This statement fails (affected rows = 0), if anyone else already updated
the row. InfoLog and Calendar then go back to the user and ask him to
manually merge his changes in the updated entry. GroupDAV/CalDAV simply
fail with “precondition fail” and it’s up to the client to deal with it.

Ralf

Raphael Alla schrieb:

Hi Ralf,

thanks for this answer. I did not realise that some apps already had a
way to handle the ‘last wins’ situation, I’ll check that.

The other issue, multiple sql updates, seem to occur a lot in my code so
I was more aware of it and took the necessary steps to make sure I used
transactions where needed. At this stage, I believe the issue should not
be addressed on a case by case basis but by allowing egroupware to
better define its data model by:

allowing to define in which context the multiple tables or updates
take place (typically when you have two objects connected by a 1 to 1, 1
to n, or n to n relationship)
improve/extend the existing so_sql class to handle these situations
as they should

I can propose some code that does the above; and I believe these issues
have been thought about in the design of Tine2. Yet at the same time I
wonder if either using existing libraries or designs from other open
source project may not be a better approach (why reinvent the wheel, but
will the wheel of other projects fit egroupware?)

A generic approach for the issue of ‘the last wins’ should be looked at
as well.

2008/2/27, Ralf Becker <RalfBecker@outdoor-training.de
mailto:RalfBecker@outdoor-training.de>:

Hi Raphael,

eGW already deals with the last wins situation in the followig apps:
- infolog (since 1.2)
- calendar (in trunk)
- groupdav (1.4)

Though only calendar uses a real counter, the other two use the last
modification date at the moment.

Data integrety or transactions is only a problem, if you have multiple
tables (or updates), as a single sql statement always runs as
transaction.

That means it's currently only critical with the calendar, where we
should add transactions around multiple table/statement updates.
I want to add a real update counter to addressbook and infolog too, as I
need them as etags for GroupDAV and CalDAV.

Ralf

Raphael Alla schrieb:

> In my opinion, 'the last wins' can be an issue, but its not so bad as
> long as you ensure that the integrity of the data remains. The real
> issue comes when there is a serie of database updates, that are
> happening simultaneously, and a race condition may lead to the data
> losing its integrity.
>
> In Blueerp (a set of modules for egw), this is handled by using sql
> transactions.
>
> However, what is not handled is the 'the last wins' issue, because in
> reality it almost never happens, and that with transactions the data
> always stays coherent.
>

<mailto:globalclown@web.de <mailto:globalclown@web.de>>>:

>
>
>     hi devs,
>     until now i was worried about small lacks of quality in
egw1.x, but
>     now i
>     read in an interview:
>
>     The most problems (with egw) occur if there are simultaneous
>     accesses to the
>     datasets.
>     In egw "the last wins".
>
>     until now, i thought, concurrent update is handled by sql - so why
>     does not
>     work in egw?
>     until now, i had no courage to make installation for huge
amount of
>     users
>     with egw,
>     so i can not check wether concurrent update works - or last wins -
>     as stated
>     above.
>     i will test it soon next weeks - that might be a nuclear bomb for
>     all user
>     data.
>
>     this is a question concerning life or death for users data and
infos,
>     so please all devs give true and reliable infos on this
problem for egw.
>     wolf
>
>     ps: such possible lack of integrity might cause big loss of
confidence.
>     --
>     View this message in context:
>    
http://www.nabble.com/concurrent-updates---in-egw-the-last-wins-----tp15567236s3741p15567236.html
>     Sent from the egroupware-developers mailing list archive at

>     Nabble.com <http://Nabble.com> <http://Nabble.com>.

>
>
>    
-------------------------------------------------------------------------
>     This SF.net email is sponsored by: Microsoft
>     Defy all challenges. Microsoft(R) Visual Studio 2008.
>     http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>     _______________________________________________
>     eGroupWare-developers mailing list
>     eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>

>     <mailto:eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>>

>     https://lists.sourceforge.net/lists/listinfo/egroupware-developers
>
>
>
>
> --
> Raphael Alla
> Mitija Australia
> +61 4 15 678 576
> +687 78 86 86
> http://www.mitija.com
>
>

>
------------------------------------------------------------------------

>
>
-------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>
>
>
------------------------------------------------------------------------
>
> _______________________________________________
> eGroupWare-developers mailing list
> eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/egroupware-developers

--

Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de
<http://www.outdoor-training.de>]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

Raphael_Alla-2 2008-02-27 12:11:45 UTC #5

Hi Raphael,

so_sql does not have the problem, as it works unmodified/un-extended
only with one table. It’s the derived classes, which eg. use multiple
so_sql instances to deal with multiple tables, who need to start the
transation.

That’s where my issue is: I find so_sql a bit too simplistic. For me, I
manipulate objects, which sometimes can be complex and have a lot of related
objects: these relations are part of my data model. These objects may be
able to be saved in a given table, or may require more than one table. I
think handling these issues belong to the ‘so’ level (the model level), so
it seems appropriate for so_sql to be able to handle a common number of
relations between objects without the user having to redo the work each
time.

The best way to deal with the last update win situation is a query like:

UPDATE table SET etag=etag+1, WHERE id=… AND etag=123

This statement fails (affected rows = 0), if anyone else already updated
the row. InfoLog and Calendar then go back to the user and ask him to
manually merge his changes in the updated entry. GroupDAV/CalDAV simply
fail with “precondition fail” and it’s up to the client to deal with it.

I understand better now. Thanks.

Ralf

Raphael Alla schrieb:

Hi Ralf,

thanks for this answer. I did not realise that some apps already had a
way to handle the ‘last wins’ situation, I’ll check that.

The other issue, multiple sql updates, seem to occur a lot in my code so
I was more aware of it and took the necessary steps to make sure I used
transactions where needed. At this stage, I believe the issue should not
be addressed on a case by case basis but by allowing egroupware to
better define its data model by:

allowing to define in which context the multiple tables or updates
take place (typically when you have two objects connected by a 1 to 1, 1
to n, or n to n relationship)

improve/extend the existing so_sql class to handle these situations
as they should

I can propose some code that does the above; and I believe these issues
have been thought about in the design of Tine2. Yet at the same time I
wonder if either using existing libraries or designs from other open
source project may not be a better approach (why reinvent the wheel, but
will the wheel of other projects fit egroupware?)

A generic approach for the issue of ‘the last wins’ should be looked at
as well.

mailto:RalfBecker@outdoor-training.de>:
Hi Raphael,

eGW already deals with the last wins situation in the followig apps:
- infolog (since 1.2)
- calendar (in trunk)
- groupdav (1.4)

Though only calendar uses a real counter, the other two use the last
modification date at the moment.

Data integrety or transactions is only a problem, if you have
multiple
tables (or updates), as a single sql statement always runs as
transaction.

That means it's currently only critical with the calendar, where we
should add transactions around multiple table/statement updates.
I want to add a real update counter to addressbook and infolog too,
as I
need them as etags for GroupDAV and CalDAV.

Ralf

Raphael Alla schrieb:

> In my opinion, 'the last wins' can be an issue, but its not so bad
as
> long as you ensure that the integrity of the data remains. The
real
> issue comes when there is a serie of database updates, that are
> happening simultaneously, and a race condition may lead to the
data
> losing its integrity.
>
> In Blueerp (a set of modules for egw), this is handled by using
sql
> transactions.
>
> However, what is not handled is the 'the last wins' issue, because
in
> reality it almost never happens, and that with transactions the
data
> always stays coherent.
>
<mailto:globalclown@web.de <mailto:globalclown@web.de>>>:
>
>
>     hi devs,
>     until now i was worried about small lacks of quality in
egw1.x, but
>     now i
>     read in an interview:
>
>     The most problems (with egw) occur if there are simultaneous
>     accesses to the
>     datasets.
>     In egw "the last wins".
>
>     until now, i thought, concurrent update is handled by sql - so
why
>     does not
>     work in egw?
>     until now, i had no courage to make installation for huge
amount of
>     users
>     with egw,
>     so i can not check wether concurrent update works - or last
wins -
>     as stated
>     above.
>     i will test it soon next weeks - that might be a nuclear bomb
for
>     all user
>     data.
>
>     this is a question concerning life or death for users data and
infos,
>     so please all devs give true and reliable infos on this
problem for egw.
>     wolf
>
>     ps: such possible lack of integrity might cause big loss of
confidence.
>     --
>     View this message in context:
>
http://www.nabble.com/concurrent-updates---in-egw-the-last-wins-----tp15567236s3741p15567236.html
>     Sent from the egroupware-developers mailing list archive at
>     Nabble.com <http://Nabble.com> <http://Nabble.com>.
>
>
>
>     This SF.net email is sponsored by: Microsoft
>     Defy all challenges. Microsoft(R) Visual Studio 2008.
>     http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>     _______________________________________________
>     eGroupWare-developers mailing list
>     eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>

>     <mailto:eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>>

>
https://lists.sourceforge.net/lists/listinfo/egroupware-developers
>
>
>
>
> --
> Raphael Alla
> Mitija Australia
> +61 4 15 678 576
> +687 78 86 86
> http://www.mitija.com
>
>

>
>
>
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>
>
>
>
> _______________________________________________
> eGroupWare-developers mailing list
> eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/egroupware-developers

--
Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de
<http://www.outdoor-training.de>]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
<mailto:eGroupWare-developers@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/egroupware-developers
–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

ManfredU 2008-02-27 12:33:55 UTC #6

Hi Raphael,

However, what is not handled is the ‘the last wins’ issue, because in
reality it almost never happens,

That depends on the application and the number of users working at the
same time. I find such a behaviour unexceptable.

Manfred

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

Patrick_Bihan-Faou-4 2008-02-27 16:33:27 UTC #7

Ralf Becker a écrit :

Hi Raphael,

so_sql does not have the problem, as it works unmodified/un-extended
only with one table. It’s the derived classes, which eg. use multiple
so_sql instances to deal with multiple tables, who need to start the
transation.

The best way to deal with the last update win situation is a query like:

UPDATE table SET etag=etag+1, WHERE id=… AND etag=123

[…]

The table ‘egw_api_content_history’ already contains some usefull
information to detect the state of various objects in the DB. Apparently
it is fairly consistently used in calendar/addressbook/infolog. Adding a
"generation" field to this table would be trivial and provide immediate
benefits to all 3 applications that are currently using this.

Are you basing your work on this or are you more or less duplicating
this functionnality in each application ?

I don’t really have an opinion on how this should be done and where the
data should be stored, but it would be nice to provide some sort of
generic ways to do this.

The reason I am interested in this is that it also affects SyncML of
course…

Patrick.

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

wolf.fle 2008-02-27 18:13:57 UTC #8

hi,
in mainframe programming, since forty years all data records for one whole transaction are at the beginning reserved for rewrite and finally all records are released or updated only, when the whole transaction is cancelled or finished.

i can not think any data processing may work in another way.
wolf

wolf.fle 2008-02-27 18:22:09 UTC #9

all records to be updated by a whole transaction must be reserved for rewrite for a special amount of time (maybe five minutes or configureable for each app) and nobody else may change these records during this time (gets a message please-wait-try-later).
wolf

Ralf_Becker_6 2008-02-27 19:56:38 UTC #10

Hi Patrick,

using the content history table has one hugh drawback. If you dont have
transactions, you run immediately in the problem you are trying to
solve: data-integrity.

I think that need to be handled by the application itself - though doing
so in shared code (eg. via so_sql) makes a lot sense.

I will have a look and implement it for the standard apps, once I’m back
home. It’s not that much work, but it requires a database schema update
and we dont want that in the released version (1.4).

Ralf

Patrick Bihan-Faou schrieb:

Ralf Becker a écrit :

Hi Raphael,

so_sql does not have the problem, as it works unmodified/un-extended
only with one table. It’s the derived classes, which eg. use multiple
so_sql instances to deal with multiple tables, who need to start the
transation.

The best way to deal with the last update win situation is a query like:

UPDATE table SET etag=etag+1, WHERE id=… AND etag=123

[…]

The table ‘egw_api_content_history’ already contains some usefull
information to detect the state of various objects in the DB. Apparently
it is fairly consistently used in calendar/addressbook/infolog. Adding a
"generation" field to this table would be trivial and provide immediate
benefits to all 3 applications that are currently using this.

Are you basing your work on this or are you more or less duplicating
this functionnality in each application ?

I don’t really have an opinion on how this should be done and where the
data should be stored, but it would be nice to provide some sort of
generic ways to do this.

The reason I am interested in this is that it also affects SyncML of
course…

Patrick.

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

Raphael_Alla-2 2008-02-28 01:22:49 UTC #11

Hi Patrick,

using the content history table has one hugh drawback. If you dont have
transactions, you run immediately in the problem you are trying to
solve: data-integrity.

I think that need to be handled by the application itself - though doing
so in shared code (eg. via so_sql) makes a lot sense.

I will have a look and implement it for the standard apps, once I’m back
home. It’s not that much work, but it requires a database schema update
and we dont want that in the released version (1.4).

Do you mean the ‘last update wins’ issue or the handling of transactions for
complex sql queries?

Ralf

Patrick Bihan-Faou schrieb:

Ralf Becker a écrit :

Hi Raphael,

so_sql does not have the problem, as it works unmodified/un-extended
only with one table. It’s the derived classes, which eg. use multiple
so_sql instances to deal with multiple tables, who need to start the
transation.

The best way to deal with the last update win situation is a query
like:

UPDATE table SET etag=etag+1, WHERE id=… AND etag=123

[…]

The table ‘egw_api_content_history’ already contains some usefull
information to detect the state of various objects in the DB. Apparently
it is fairly consistently used in calendar/addressbook/infolog. Adding a
"generation" field to this table would be trivial and provide immediate
benefits to all 3 applications that are currently using this.

Are you basing your work on this or are you more or less duplicating
this functionnality in each application ?

I don’t really have an opinion on how this should be done and where the
data should be stored, but it would be nice to provide some sort of
generic ways to do this.

The reason I am interested in this is that it also affects SyncML of
course…

Patrick.

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

ManfredU 2008-02-28 07:47:22 UTC #12

Hi Wolf,

pessimistic locking doesn’t work with web apps. You can’t keep the
records / table space etc. locked that long. You don’t know if the
users even responds at all. Other users would have to wait until the
lock expires and they can not access the data during that time.

The way to do it is exactly as Ralf has explained: Read timestamp or
record version und use the old version in the update and update it at
the sametime.

See

http://en.wikipedia.org/wiki/Timestamp-based_concurrency_control
http://en.wikipedia.org/wiki/Multiversion_concurrency_control
http://en.wikipedia.org/wiki/Non-lock_concurrency_control
http://en.wikipedia.org/wiki/Optimistic_locking.

Even with our large scale mainframe application (CICS, Cobol, DB2) we
use the timestamp method successfully since nearly 20 years.

Cheers,
Manfred

hi,
in mainframe programming, since forty years all data records for one
whole transaction are at the beginning reserved for rewrite and
finally all records are released or updated only, when the whole
transaction is cancelled or finished.

i can not think any data processing may work in another way.
wolf

–

Manfred Usselmann usselmann.m@icg-online.de

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

ManfredU 2008-02-28 08:23:52 UTC #13

No, not feasible for eGW or other web apps! Locking does not work there
because of the stateless nature of the HTTP protocol. There are
different methods available, see my other mail.

Manfred

all records to be updated by a whole transaction must be reserved for
rewrite for a special amount of time (maybe five minutes or
configureable for each app) and nobody else may change these records
during this time (gets a message please-wait-try-later).
wolf

–
View this message in context:
http://www.nabble.com/Re%3A-concurrent-updates---in-egw-the-last-wins-----tp15710432s3741p15719577.html
Sent from the egroupware-developers mailing list archive at
Nabble.com.

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–

Manfred Usselmann usselmann.m@icg-online.de

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

Ralf_Becker_6 2008-02-28 10:51:46 UTC #14

Hi,

I was thinking both, it’s not that much to do.

Ralf

“Raphael Alla” raphael.alla@gmail.com schrieb:

2008/2/28, Ralf Becker <RalfBecker@outdoor-training.de>:Hi Patrick,

using the content history table has one hugh drawback. If you dont have
transactions, you run immediately in the problem you are trying to
solve: data-integrity.

I think that need to be handled by the application itself - though doing
so in shared code (eg. via so_sql) makes a lot sense.

I will have a look and implement it for the standard apps, once I’m back
home. It’s not that much work, but it requires a database schema update
and we dont want that in the released version (1.4).

Do you mean the ‘last update wins’ issue or the handling of transactions
for complex sql queries?

Ralf
Patrick Bihan-Faou schrieb:

Ralf Becker a écrit :

Hi Raphael,

so_sql does not have the problem, as it works unmodified/un-extended
only with one table. It’s the derived classes, which eg. use multiple
so_sql instances to deal with multiple tables, who need to start the
transation.

The best way to deal with the last update win situation is a query like:

UPDATE table SET etag=etag+1, WHERE id=… AND etag=123

[…]

The table ‘egw_api_content_history’ already contains some usefull
information to detect the state of various objects in the DB. Apparently
it is fairly consistently used in calendar/addressbook/infolog. Adding a
"generation" field to this table would be trivial and provide immediate
benefits to all 3 applications that are currently using this.

Are you basing your work on this or are you more or less duplicating
this functionnality in each application ?

I don’t really have an opinion on how this should be done and where the
data should be stored, but it would be nice to provide some sort of
generic ways to do this.

The reason I am interested in this is that it also affects SyncML of
course…

Patrick.

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
– eGroupWare Training & Support ==> http://www.egroupware-support.deOutdoor
Unlimited Training GmbH [http://www.outdoor-training.de]Geschäftsführer:
Ralf und Birgit BeckerLeibnizstr. 1767663 Kaiserslautern

Raphael_Alla-2 2008-03-06 05:11:47 UTC #15

Hi,

I was thinking both, it’s not that much to do.

Great, but then we must think as to how egroupware will know when there is a
complex data structure or not. For me this should come from allowing
egroupware to define data models more thoroughly than the current
capabilities of dbtools. I have already expressed my ideas on this one.

Ralf

“Raphael Alla” raphael.alla@gmail.com schrieb:

Hi Patrick,

using the content history table has one hugh drawback. If you dont have
transactions, you run immediately in the problem you are trying to
solve: data-integrity.

I think that need to be handled by the application itself - though doing
so in shared code (eg. via so_sql) makes a lot sense.

I will have a look and implement it for the standard apps, once I’m back
home. It’s not that much work, but it requires a database schema update
and we dont want that in the released version (1.4).

Do you mean the ‘last update wins’ issue or the handling of transactions
for complex sql queries?

Ralf

Patrick Bihan-Faou schrieb:

Ralf Becker a écrit :

Hi Raphael,

so_sql does not have the problem, as it works unmodified/un-extended
only with one table. It’s the derived classes, which eg. use multiple
so_sql instances to deal with multiple tables, who need to start the
transation.

The best way to deal with the last update win situation is a query
like:

UPDATE table SET etag=etag+1, WHERE id=… AND
etag=123

[…]

The table ‘egw_api_content_history’ already contains some usefull
information to detect the state of various objects in the DB.
Apparently
it is fairly consistently used in calendar/addressbook/infolog. Adding
a
"generation" field to this table would be trivial and provide
immediate
benefits to all 3 applications that are currently using this.

Are you basing your work on this or are you more or less duplicating
this functionnality in each application ?

I don’t really have an opinion on how this should be done and where
the
data should be stored, but it would be nice to provide some sort of
generic ways to do this.

The reason I am interested in this is that it also affects SyncML of
course…

Patrick.

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [http://www.outdoor-training.de]
Geschäftsführer: Ralf und Birgit Becker
Leibnizstr. 17
67663 Kaiserslautern

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

Ralf_Becker_6 2008-03-06 06:57:11 UTC #16

Hi Raphael,

Raphael Alla schrieb:

2008/2/28, Ralf Becker <RalfBecker@outdoor-training.de
mailto:RalfBecker@outdoor-training.de>:
Hi,

I was thinking both, it's not that much to do.
Great, but then we must think as to how egroupware will know when there
is a complex data structure or not. For me this should come from
allowing egroupware to define data models more thoroughly than the
current capabilities of dbtools.

At first it’s of cause the responsibility of the app
(developer/maintainer) to use transactions, if he has a multiple table
data model, or does other db-actions using more then one query.

If we update db_tools/so_sql to handle multiple table data models, we
have to add it there too.

I have already expressed my ideas on this one.

I saw your posts while I was in Australia, but I had not time to read
them. If you want to work on this feel free to do so. The only
requirement I have is, that the current code using so_sql/db_tools
continue to work (with as little modification as possible) and of cause
at least at current speed.

Same is true with your table-widget. I think I also seen posts
concerning that. If you need help with that let me know (again, as I
dont have the time to read the whole list from the last months).

Ralf

Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

Raphael_Alla-2 2008-03-12 06:53:44 UTC #17

It came to my attention that supporting transactions in egroupware will
require the setup to be changed as well so that people can choose the
’mysqlt’ driver for adodb rather than the default mysql which does not
support transactions.

I propose for this driver to be the default, unless we have evidence of
degraded performance when using it.

Let me know what you think.

Cheers

Raphael

Hi Raphael,

Raphael Alla schrieb:

mailto:RalfBecker@outdoor-training.de>:
Hi,

I was thinking both, it's not that much to do.
Great, but then we must think as to how egroupware will know when there
is a complex data structure or not. For me this should come from
allowing egroupware to define data models more thoroughly than the
current capabilities of dbtools.
At first it’s of cause the responsibility of the app
(developer/maintainer) to use transactions, if he has a multiple table
data model, or does other db-actions using more then one query.

If we update db_tools/so_sql to handle multiple table data models, we
have to add it there too.

I have already expressed my ideas on this one.

I saw your posts while I was in Australia, but I had not time to read
them. If you want to work on this feel free to do so. The only
requirement I have is, that the current code using so_sql/db_tools
continue to work (with as little modification as possible) and of cause
at least at current speed.

Same is true with your table-widget. I think I also seen posts
concerning that. If you need help with that let me know (again, as I
dont have the time to read the whole list from the last months).

Ralf

–

Ralf Becker
eGroupWare Training & Support ==> http://www.egroupware-support.de
Outdoor Unlimited Training GmbH [www.outdoor-training.de]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 (0)631 31657-0

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers

–
Raphael Alla
Mitija Australia
+61 4 15 678 576
+687 78 86 86
http://www.mitija.com

Nathan_Gray-2 2008-03-12 14:05:50 UTC #18

…
I propose for this driver to be the default, unless we have evidence
of degraded performance when using it.

Let me know what you think.

Yes. If possible, transactions by default would be really nice.

Nathan Gray
nathan at goarctic dot com

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft® Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

eGroupWare-developers mailing list
eGroupWare-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/egroupware-developers