Skip to main content
Legend
January 20, 2021
Question

Continuation of "Ever increasing number of database connections with JDBC and ODBC drivers - MSSQL"

  • January 20, 2021
  • 2 replies
  • 2681 views

Hi,

 

This is a continuation of the thread: https://community.adobe.com/t5/coldfusion/ever-increasing-number-of-database-connections-with-jdbc-and-odbc-drivers-mssql/m-p/11748504?page=1

 

For some reason I cannot post to that thread any longer, so am creating this one to carry on.

 

    This topic has been closed for replies.

    2 replies

    tribuleAuthor
    Legend
    January 20, 2021

    I took up a kind offer of help from Charlie and we tried a new JNDI data source. JNDI can be configured to use TomCat's own connection pooling (so as to "bypass" JDBCs own pooling system). This worked for a while, but then as the server got busier (as our back-end office staff came on line) the same problem was experienced. Connections rose and JDBC connections went to zero. The server could not serve any pages and had to be restarted. Changing various settings on the server did not help. No solution has been found so far.

    BKBK
    Community Expert
    Community Expert
    January 20, 2021

    Recap:

    @tribule , In our e-mail correspondence, I suggested the following, as a possible fix: replace multiple instantiations of the database object with just one. For example, instead of

     

        x1=objToken.getcnnSG().dbFunction1();
        x2=objToken.getcnnSG().dbFunction2();
        objToken.getcnnSG().dbFunction3("some sql string arg");

     

    instantiate the database object just once, using

     

    // the single instantiation
    dbObject=objToken.getcnnSG();

     

    This guarantees that every function will be called on the same object. Thus, following dbObject.query(), the call dbOject.ReleaseConnection() will release the connection that dbObject used.

     

    // use the instance in subsequent calls

    x1=dbObject.dbFunction1();
    x2=dbObject.dbFunction2();
    dbObject.dbFunction3("some sql string arg");

     

    // also use the instance to interact with the connection
    dbObject.reset()
    /* Alternatively: dbObject.ReleaseConnection(); */
    dbObject.setConnectString("max pool size=100");

     

    Follow-up:

    1) Use var for every variable defined in your custom functions. This ensures that resources will be released, closed or garbage-collected.

    2) When the database object has finished its job (querying, interacting, releasing the connection, etc.), then kill the object itself. That is, set it to null. To do so, use, for example:

     

    dbObject= javacast("null", 0);

     

    The association between the database object and its constituent Statements, Connections, DSN, etc is one of composition. Therefore, when the database object becomes null, so, too, will each of its constituents.

     

    tribuleAuthor
    Legend
    January 21, 2021

    Thanks. I have added var's and releaseconnection() and the nullification of the variable.

     

    At this point in time there are 21000 connections being used in the master.dbo.sysprocesses table. Will be interesting to see what happens when it reaches 32K! (MSSQL limit).

     

    I noticed that MS say that master.dbo.sysprocessess is now deprecated and that we should use 

    sys.dm_exec_connections instead. If I use the latter I still see the same number of connections. I also see that the "packet size" is 8000 bytes. Is that ok - I think it's the JDBC default?

     

    BKBK
    Community Expert
    Community Expert
    January 20, 2021

    @tribule, I am glad to see that you can now resume the thread. You and I have corresponded privately on it. You shared with me the Java code that is causing the problem.

     

    The code enables the application to create connections to the database. My hypothesis is that:

     

    1) the code may be creating connections indefinitely;

    and/or

    2) there may be inadequate or no connection pooling;

    and/or

    3) connections are not being closed.

    tribuleAuthor
    Legend
    January 20, 2021

    Hi. To give context to others, from what you saw, the code in the Java database class was properly opening and closing connections, however the cfscript code that used that class may have been incorrectly coded in so far as it was not apparant if the calls to the database (to execute SQL) were properly opening and closing just one connection at a time. @BKBK  if you want to explain that better, please do 🙂


    I have now made modifications to a lot of templates that had database calls, and am testing to see if the system behaves any better.  Just to reiterate, we are using this MSSQL query to check how many connections to the server are occuring. This is the query which previously has yielded more and more results - it never frees up any rows:

    SELECT *
    FROM master.dbo.sysprocesses(NOLOCK) 
    WHERE (spid > 50) AND (status = 'sleeping') AND (program_name LIKE '%jdbc%')

     

    I am not sure if this query should yield a larger and larger result set, or if it should get smaller (automatic data purge by MSSQL) at some point? If someone knows that, I'd appreciate your input.

    I will update again soon after watching the server. I do hope, as @BKBK says, that the code may be the problem, but so far this remains unknown

    Regards,

    Mark

    Charlie Arehart
    Community Expert
    Community Expert
    January 20, 2021

    Quick update, the master.dbo.sysprocesses table now shows nearly 7000 connections, and that is 2 hours 20 minutes from restarting the server. I think we have a maximum of 32K in MSSQL? Not sure what will happen, but MSSQL does not seem to be housekeeping all of these sleeping connections, so far.


    Mark, that's encouraging, sure. It would be interesting also to hear if FR's tracking of connections (in Metrics>CF Metrics) is also showing the conneciton count staying down, compared to before.

     

    And if this continues, even with your back-end office staff having come on line, the question then will be what change you did that made the difference. Your last comment said, "I have now made modifications to a lot of templates that had database calls, and am testing to see if the system behaves any better." Any chance you may be able to identify readily which change seemed to do the trick.

     

    This has been a knotty problem. I'd not updated the post here (or the last one) with any of the many diagnostic and configuration approaches we'd attempted. I was always leary to propose any code changes, not knowing anything about your app--and also fearful that "just trying things" might not be fruitful. I was really hopeful that some diagnostic may have pointed out WHERE the problem was, or that perhaps some configuration change may have "covered it".

     

    Since neither was fruitful, and since the problem continued, I understand why you'd resort to just trying various code changes until something seemed to do the trick. Looking forward to hear what that was. We have sensed from the beginning that it was some transaction that was somehow being left "open", but it's very hard to find that needle in a haystack of thousands of lines of code. (And to be clear to anyone interested, I did not charge for the time, which was many dozens of hours the past few weeks. Besides my initial offer in Dec to help for free, I also have a satisfaction guarantee whereby I don't expect to get paid for my time if it proves of no value.)

     

    Looking forward to hearing how things turn out, or if the mystery continues, in which case we'll keep digging.

    /Charlie (troubleshooter, carehart. org)