Any OOP experts in here?

> If the file is opened and locked by 1 process for writing, then the file
> can't be accessed by another process for reading - you'll get an
> IOException.
>
> Have you covered that scenario?

Well, again, goes back to maybe using the DB for this. ;o)

The more I try to deal with this problem, the more I come to the conclusion
that the filesystem just isn't made for this it seems.

One suggest that I got, and may try was to do the following:

set up a looping counter field in the database with valuse from, say, 0-9

Then, to write my menu.xml file:
- first query to get the current count (let's use '5')
- set the count to 6
- write out menu5.xml


The idea was that no two processes should end up writing the same xml file.

The drawback is that I can't just cache one file on the front end now and,
instead, I'd have to check for th emost recent menu#.xml file each and every
page request.

> For instance, if admin1 submits an update and that locks the file for 2
> seconds. If a user comes to the site at that same time, they'll get an
> IOException cause the file is locked.

Ah, that part is handled a bit differently. We have a staging server and a
public server, so the live site is never actually hitting the server that is
being written to. However, the conflict is still there if we want to view
the site on the staging server.


> If using the XML file as a source of data turns out to be a problem, then
> consider using the Cache to store the common dataset, and update the
> dataset in the cache versus locking and updating a file.

Yes...exactly! I think you're echoing my thinking too. Seems like the DB is
the better way to manage all the threading issues and skip the XML file
altogether!

-Darrel

The following might provide a solution for allowing read access to a file
that is currently locked.

1. Create the FileStream with FileShare.Read when the process wants a write
lock on the file
2. Create The FileStream with FileShare.Write when the process wants a read
lock on the file
3. Use a ReaderWriterLock instance instead of SyncLock when writing to the
file:

class MenuWriter
{
private static ReaderWriterLock writerLock = new ReaderWriterLock();

public static void Write()
{
try
{
// acquire write lock, specify in milliseconds how long to wait
writerLock.AcquireWriterLock( -1 ); // -1 = waits forever for
the lock
FileStream fs = new FileStream( path, FileMode.Create,
FileAccess.Write, FileShare.Read );
try
{
// write to file
}
catch
{
// exception writing to file
}
finally
{
fs.Close();
writerLock.ReleaseWriterLock();
}
}
catch
{
// exception acquiring lock
}
}

4. Use ReaderWriterLock to get a read lock from the file

// webpage.aspx.cs
void Page_Load()
{
ReaderWriterLock readerLock = new ReaderWriterLock();
try
{
readerLock.AcquireReaderLock( -1 ); // infinite, or page timeout
FileStream fs = new FileStream( ..., ..., ...,
FileShare.Write );
try
{
// read from file
}
catch
{
// exception reading file
}
finally
{
fs.Close();
readerLock.ReleaseReaderLock();
}
}
catch
{
// exception acquiring lock
}
}

Ron

"RYoung" <ryoungdev_@gmail.com> wrote in message
news:e52jeq$r4h$1@forums.macromedia.com...
>I was just thinking of something.
>
> If the file is opened and locked by 1 process for writing, then the file
> can't be accessed by another process for reading - you'll get an
> IOException.
>

I'm not quite sure of the ASP.NET worker process but I believe it has an
available pool of threads to use for handling web requests.

If 10 admins were to update the page within milliseconds of each other then
essentially 10 threads would try to acquire that physical text file for
writing. If no SyncLock were in place, then there would be IOExceptions.
With the SyncLock each thread will have to wait for the previous thread to
release the lock so it can acquire it, write to the file, release the lock,
and so on.

Initially I thought the page would timeout if 10 threads waited 2 seconds
each for a lock, meaning the last thread would wait 20 seconds. I tried it
with 2 threads in an ASP.NET page with a 20 second delay and it didn't
timeout. Perhaps it would if there were a 60 second delay. But that would
only be the case if the method took an extremely long amount of time, or a
large number of admins submitted simulataneously - both unlikely cases.

Essentially, waiting for the SyncLock is just the threads waiting in line
for their turn - or queing up.

Ron

"darrel" <notreal@nowhere.com> wrote in message
news:e52j6c$qom$1@forums.macromedia.com...
>> Also note that Thead.Sleep(2000) is just to simulate a process that takes
>> 2 seconds and give me a change to see the threads queue in line waiting
>> for the lock.
>
> So you CAN queue threads!? This is the first I've been told that you can
> do this. Most advice has been to drop the filesystem altogether and store
> the XML in the DB instead to let the DB handle the Queue. Interesting!
> Thanks for the in-depth analysis, Ron. I'll definitely be taking a look at
> this.
>
> -Darrel
>

I was just thinking of something.

If the file is opened and locked by 1 process for writing, then the file
can't be accessed by another process for reading - you'll get an
IOException.

Have you covered that scenario?

For instance, if admin1 submits an update and that locks the file for 2
seconds. If a user comes to the site at that same time, they'll get an
IOException cause the file is locked.

If using the XML file as a source of data turns out to be a problem, then
consider using the Cache to store the common dataset, and update the dataset
in the cache versus locking and updating a file.

Ron

"darrel" <notreal@nowhere.com> wrote in message
news:e529b2$eqv$1@forums.macromedia.com...
>
>> What's the problem?
>
> I'll try to explain. ;o)
>
> I have an abridged version of the funtion at the bottom of this post if
> you want to take a look at it.
>
> The scenario:
>
> Our CMS, whenever a page is updated on the back end, calls a function that
> then queries the DB recursively to generate an XML file of all the data.
> The intent was that the XML file is easy to cache on the front-end, so we
> can use it to store all the page-centric variables and to generate all the
> site menus without having to make additional hits on the DB on each page
> load.
>
> The problem is that I THINK we're having threading issues on the back end.
> As more people use the CMS, we're finding more errors that we think are
> related to the XML file function either being 'interupted' by another
> call, or being locked in the filesystem itself. I'm not sure if it's bad
> code on my part, or the Windows File system just not handling things very
> well.
>
> At this point, I'm just out of energy to fight it anymore. I think I may
> just store the XML back into the DB and let the DB handle all the
> threading/locking issues for me.
>
> That said, I'd love to know what, specifically, I'm doing wrong, so if you
> see anyting amiss, please let me know!
>
> -Darrel
>
>
> My class file:
> -------------------------------------------------------------
> Imports System.IO
> Public Class menuWriter
>
> Shared fs As System.IO.FileStream
>
> Shared Sub createXMLfileWithXmlWriter(ByVal parentNode As Integer,
> ByVal intLevel As Integer, ByVal rowCount As Integer, ByVal
> createParentNode As Boolean, ByVal siteID As Integer)
>
> Try
>
> Dim objXMLWriter As System.Xml.XmlTextWriter
> Dim path As String =
> System.Web.HttpContext.Current.Server.MapPath("/mjb05/xml/" & siteID &
> "/siteMenu.xml").ToString
> fs = New FileStream(path, FileMode.Create, FileAccess.Write,
> FileShare.None)
> SyncLock fs
> objXMLWriter = New System.xml.XmlTextWriter(fs,
> System.Text.Encoding.Default)
> objXMLWriter.Formatting = objXMLWriter.Formatting.Indented
> objXMLWriter.Indentation = 3
> objXMLWriter.WriteStartDocument()
> objXMLWriter.WriteComment("Created on " & Now())
> objXMLWriter.WriteStartElement("menuItems")
>
> WriteXmlData(objXMLWriter, parentNode, siteID)
>
> objXMLWriter.WriteEndElement() 'close menuItems node
> objXMLWriter.WriteEndDocument() 'close docment
> objXMLWriter.Flush()
> objXMLWriter.Close()
> End SyncLock
> Catch ex As Exception
> ...catch errors
> Finally
> fs.Close()
> End Try
>
> End Sub
>
>
> Shared Sub WriteXmlData(ByVal objXMLWriter As System.Xml.XmlTextWriter,
> ByVal parentNode As Integer, ByVal siteID As Integer, Optional ByVal
> intLevel As Int32 = 0)
>
> 'retrieve all children of parentNode
> ' grab the data from the db
> Dim DS As New DataSet
> ...set up DB connection
> objOleDbAdapter.Fill(DS, "webPages")
>
>
> ' Walk through results whether it's a list or a single contact.
> ' based off of this tutorial:
> '
> http://www.wwwcoder.com/main/parentid/154/site/899/68/default.aspx
>
> Dim rowCount As Int32 = 0
> While rowCount < DS.Tables(0).Rows.Count 'ie, if there IS
> data, then...
> objXMLWriter.WriteStartElement("menuItem")
>
>
> objXMLWriter.WriteElementString("pageID",
> Trim(DS.Tables(0).Rows(rowCount)("pageID").ToString))
> objXMLWriter.WriteElementString("parentID",
> Trim(DS.Tables(0).Rows(rowCount)("parentID").ToString))
> ...write rest of elements.
>
> 'now call the subroutine we're in to see if this value has
> 'any children and increase the indent, and so on...
> WriteXmlData(objXMLWriter,
> Convert.ToInt32(DS.Tables(0).Rows(rowCount).Item(0)), siteID, intLevel +
> 1)
> rowCount = rowCount + 1
> objXMLWriter.WriteEndElement() 'close menuItem
> node
> End While
>
> ' clean up
>
> objConnect.Close()
> objOleDbAdapter.Dispose()
> objCommand.Dispose()
>
> End Sub
>
> End Class
>
>

> Also note that Thead.Sleep(2000) is just to simulate a process that takes
> 2 seconds and give me a change to see the threads queue in line waiting
> for the lock.

So you CAN queue threads!? This is the first I've been told that you can do
this. Most advice has been to drop the filesystem altogether and store the
XML in the DB instead to let the DB handle the Queue. Interesting! Thanks
for the in-depth analysis, Ron. I'll definitely be taking a look at this.

-Darrel

I simulated the process and got some unexpected results.

Basically I created a MenuWriter class that has a static method Write() and
a static FileStream object "fs". In that regard it's a close implementation
of your menuWriter class.

Here is what the Write() method originally looked like:

// MenuWriter.Write - simulates 2 second operation
static void Write()
{
try
{
fs = new FileStream( @"c:\dev\file.xml", FileMode.Create,
FileAccess.Write, FileShare.None );
lock ( fs )
{
Console.WriteLine( "thread {0}",
Thread.CurrentThread.ManagedThreadId );
Thread.Sleep(2000);
}
}
catch{}
finally{ fs.Close(); }
}

The Main() method created an array of 20 threads. Each thread would start
the Write method.

void Main()
{
int max = 20;
Thread[] threads = new Thread[max];

for ( int i = 0; i < max; i++ )
{
threads = new Thread( new ThreadStart(MenuWriter.Write));
}

for ( int i = 0; i < max; i++ )
{
threads.Start();
}
}

What I expected to happen was that every 2 seconds the line "thread ##"
would be written to the console, where ## is the unique thread ID.
I expected to see this output 20 times, occuring every 2 seconds.

What happened was either:
a. Only about 7-9 lines would be written before the program terminated
without throwing an exception. There was no delay in between each line being
written.
or
b. 7-9 lines written with no delay and an NullReferenceException would occur
pointing to Line 52 which was the "fs.Close()" line in the finally block.

Even if all 20 lines were written with no exception, the lack of the 2
second delay tells me that there was no lock, or the lock is implemented
incorrectly.

So, 2 things to correct that and get the expected results:

1. Make the FileStream object local to the method.
2. Lock something other than the FileStream object.

The re-write:

class MenuWriter
{
private static object syncLock = new object(); // the object to lock

public static void Write()
{
lock ( syncLock )
{
// do all the work in here
FileStream fs = new FileStream( .... );
try
{
string message = "Written by " +
Thread.CurrentThread.ManagedId.ToString();
fs.Write(Encoding.ASCII.GetBytes(message), 0,
message.Length);
Thread.Sleep(2000);
}
catch{}
finally{ fs.Close(); }
}
}
}

With those changes, I observed 20 lines output to the console, occuring
every 20 seconds, which I believe simulates what you want to accomplish.

Additionally, I went ahead and wrote to a text file in the Write() method as
shown above. Where the console output that Thread 22 was the last thread to
get the lock, this was also written to the text file, verifying that 20
threads waited for the lock, acquired it, wrote to the file, released the
lock.

Also note that Thead.Sleep(2000) is just to simulate a process that takes 2
seconds and give me a change to see the threads queue in line waiting for
the lock.

Ron

"darrel" <notreal@nowhere.com> wrote in message
news:e529b2$eqv$1@forums.macromedia.com...
>
>> What's the problem?
>
> I'll try to explain. ;o)
>
> I have an abridged version of the funtion at the bottom of this post if
> you want to take a look at it.
>
> The scenario:
>
> Our CMS, whenever a page is updated on the back end, calls a function that
> then queries the DB recursively to generate an XML file of all the data.
> The intent was that the XML file is easy to cache on the front-end, so we
> can use it to store all the page-centric variables and to generate all the
> site menus without having to make additional hits on the DB on each page
> load.
>
> The problem is that I THINK we're having threading issues on the back end.
> As more people use the CMS, we're finding more errors that we think are
> related to the XML file function either being 'interupted' by another
> call, or being locked in the filesystem itself. I'm not sure if it's bad
> code on my part, or the Windows File system just not handling things very
> well.
>
> At this point, I'm just out of energy to fight it anymore. I think I may
> just store the XML back into the DB and let the DB handle all the
> threading/locking issues for me.
>
> That said, I'd love to know what, specifically, I'm doing wrong, so if you
> see anyting amiss, please let me know!
>
> -Darrel
>
>
> My class file:
> -------------------------------------------------------------
> Imports System.IO
> Public Class menuWriter
>
> Shared fs As System.IO.FileStream
>
> Shared Sub createXMLfileWithXmlWriter(ByVal parentNode As Integer,
> ByVal intLevel As Integer, ByVal rowCount As Integer, ByVal
> createParentNode As Boolean, ByVal siteID As Integer)
>
> Try
>
> Dim objXMLWriter As System.Xml.XmlTextWriter
> Dim path As String =
> System.Web.HttpContext.Current.Server.MapPath("/mjb05/xml/" & siteID &
> "/siteMenu.xml").ToString
> fs = New FileStream(path, FileMode.Create, FileAccess.Write,
> FileShare.None)
> SyncLock fs
> objXMLWriter = New System.xml.XmlTextWriter(fs,
> System.Text.Encoding.Default)
> objXMLWriter.Formatting = objXMLWriter.Formatting.Indented
> objXMLWriter.Indentation = 3
> objXMLWriter.WriteStartDocument()
> objXMLWriter.WriteComment("Created on " & Now())
> objXMLWriter.WriteStartElement("menuItems")
>
> WriteXmlData(objXMLWriter, parentNode, siteID)
>
> objXMLWriter.WriteEndElement() 'close menuItems node
> objXMLWriter.WriteEndDocument() 'close docment
> objXMLWriter.Flush()
> objXMLWriter.Close()
> End SyncLock
> Catch ex As Exception
> ...catch errors
> Finally
> fs.Close()
> End Try
>
> End Sub
>
>
> Shared Sub WriteXmlData(ByVal objXMLWriter As System.Xml.XmlTextWriter,
> ByVal parentNode As Integer, ByVal siteID As Integer, Optional ByVal
> intLevel As Int32 = 0)
>
> 'retrieve all children of parentNode
> ' grab the data from the db
> Dim DS As New DataSet
> ...set up DB connection
> objOleDbAdapter.Fill(DS, "webPages")
>
>
> ' Walk through results whether it's a list or a single contact.
> ' based off of this tutorial:
> '
> http://www.wwwcoder.com/main/parentid/154/site/899/68/default.aspx
>
> Dim rowCount As Int32 = 0
> While rowCount < DS.Tables(0).Rows.Count 'ie, if there IS
> data, then...
> objXMLWriter.WriteStartElement("menuItem")
>
>
> objXMLWriter.WriteElementString("pageID",
> Trim(DS.Tables(0).Rows(rowCount)("pageID").ToString))
> objXMLWriter.WriteElementString("parentID",
> Trim(DS.Tables(0).Rows(rowCount)("parentID").ToString))
> ...write rest of elements.
>
> 'now call the subroutine we're in to see if this value has
> 'any children and increase the indent, and so on...
> WriteXmlData(objXMLWriter,
> Convert.ToInt32(DS.Tables(0).Rows(rowCount).Item(0)), siteID, intLevel +
> 1)
> rowCount = rowCount + 1
> objXMLWriter.WriteEndElement() 'close menuItem
> node
> End While
>
> ' clean up
>
> objConnect.Close()
> objOleDbAdapter.Dispose()
> objCommand.Dispose()
>
> End Sub
>
> End Class
>
>

> What's the problem?

I'll try to explain. ;o)

I have an abridged version of the funtion at the bottom of this post if you
want to take a look at it.

The scenario:

Our CMS, whenever a page is updated on the back end, calls a function that
then queries the DB recursively to generate an XML file of all the data. The
intent was that the XML file is easy to cache on the front-end, so we can
use it to store all the page-centric variables and to generate all the site
menus without having to make additional hits on the DB on each page load.

The problem is that I THINK we're having threading issues on the back end.
As more people use the CMS, we're finding more errors that we think are
related to the XML file function either being 'interupted' by another call,
or being locked in the filesystem itself. I'm not sure if it's bad code on
my part, or the Windows File system just not handling things very well.

At this point, I'm just out of energy to fight it anymore. I think I may
just store the XML back into the DB and let the DB handle all the
threading/locking issues for me.

That said, I'd love to know what, specifically, I'm doing wrong, so if you
see anyting amiss, please let me know!

-Darrel


My class file:
-------------------------------------------------------------
Imports System.IO
Public Class menuWriter

Shared fs As System.IO.FileStream

Shared Sub createXMLfileWithXmlWriter(ByVal parentNode As Integer, ByVal
intLevel As Integer, ByVal rowCount As Integer, ByVal createParentNode As
Boolean, ByVal siteID As Integer)

Try

Dim objXMLWriter As System.Xml.XmlTextWriter
Dim path As String =
System.Web.HttpContext.Current.Server.MapPath("/mjb05/xml/" & siteID &
"/siteMenu.xml").ToString
fs = New FileStream(path, FileMode.Create, FileAccess.Write,
FileShare.None)
SyncLock fs
objXMLWriter = New System.xml.XmlTextWriter(fs,
System.Text.Encoding.Default)
objXMLWriter.Formatting = objXMLWriter.Formatting.Indented
objXMLWriter.Indentation = 3
objXMLWriter.WriteStartDocument()
objXMLWriter.WriteComment("Created on " & Now())
objXMLWriter.WriteStartElement("menuItems")

WriteXmlData(objXMLWriter, parentNode, siteID)

objXMLWriter.WriteEndElement()
'close menuItems node
objXMLWriter.WriteEndDocument()
'close docment
objXMLWriter.Flush()
objXMLWriter.Close()
End SyncLock
Catch ex As Exception
...catch errors
Finally
fs.Close()
End Try

End Sub


Shared Sub WriteXmlData(ByVal objXMLWriter As System.Xml.XmlTextWriter,
ByVal parentNode As Integer, ByVal siteID As Integer, Optional ByVal
intLevel As Int32 = 0)

'retrieve all children of parentNode
' grab the data from the db
Dim DS As New DataSet
...set up DB connection
objOleDbAdapter.Fill(DS, "webPages")


' Walk through results whether it's a list or a single contact.
' based off of this tutorial:
' http://www.wwwcoder.com/main/parentid/154/site/899/68/default.aspx

Dim rowCount As Int32 = 0
While rowCount < DS.Tables(0).Rows.Count 'ie, if there IS
data, then...
objXMLWriter.WriteStartElement("menuItem")


objXMLWriter.WriteElementString("pageID",
Trim(DS.Tables(0).Rows(rowCount)("pageID").ToString))
objXMLWriter.WriteElementString("parentID",
Trim(DS.Tables(0).Rows(rowCount)("parentID").ToString))
...write rest of elements.

'now call the subroutine we're in to see if this value has
'any children and increase the indent, and so on...
WriteXmlData(objXMLWriter,
Convert.ToInt32(DS.Tables(0).Rows(rowCount).Item(0)), siteID, intLevel + 1)
rowCount = rowCount + 1
objXMLWriter.WriteEndElement() 'close menuItem
node
End While

' clean up

objConnect.Close()
objOleDbAdapter.Dispose()
objCommand.Dispose()

End Sub

End Class

What's the problem?

Ron

"darrel" <notreal@nowhere.com> wrote in message
news:e5279i$c8l$1@forums.macromedia.com...
> We're hung up on a function of ours (ASP.net) that I'm pretty sure is
> suffering from thread conflicts likely due to me not fully understanding
> OOP.
>
> This probably isn't the best forum to ask it in but I'm running out of
> places to go. ;o)
>
> So, if anyone wants to take a stab at it, let me know!
>
> -Darrel
>