• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

Coldfusion 2018 clustering and session replication not working

Community Beginner ,
Sep 28, 2018 Sep 28, 2018

Copy link to clipboard

Copied

Setting up a couple new Coldfusion 2018 servers and will be using clustering for the first time and have run into some problems.

I am having trouble with session replication. Basically, session variables appear to be replicated between nodes in a cluster but are killed after a short while at random.

A little setup info:

  • 2 web servers (Windows Server 2012) behind load balancers
  • On each web server sits a Coldfusion cluster consisting of 2 local instances (still unclear if this is useful or not - will ask in separate question) and 2 remote instances (the remotes reference the local instances of each opposite server)
  • For simplicity, currently just testing on a single server with local Coldfusion instances - leaving the remotes out of the equation until I can get things working reliably locally
  • Using J2EE session variables
  • Coldfusion session timeout set to 2 hours

Here is what I did/experienced:

  • We have a web application that requires login and stores user information in the session upon login.
  • I made a small modification to the web app to show me which cluster instance has serviced my current request.
  • After setting up the cluster, I started the web application and logged in, noting the instance which displayed the login page.
  • Upon logging in, I was immediately returned to the login screen (app checks for user info in session and redirects to login if not found)
  • Debugging revealed that I was actually being logged in but after redirecting to some new page after login the user info would be gone from session.
  • Multiple login attempts in a row (same credentials, just tried over and over again and again) revealed that sometimes login would proceed just fine and I would get into the app. However, if I refreshed the page or went to another page, the session would be lost very soon but at random (within a few page refreshes).
  • In an attempt to simplify the problem to try and figure out what is going on, I created a simple .cfm that bypasses all the login stuff and does one thing: adds a simple string value to session and then dumps the session and instance name. ** I ran the script once, noted which instance was being used and that session contained my value. ** I then edited the script so it no longer set the session value. ** I then hit refresh over and over so I could confirm:
    1. That requests were being serviced by both instances in cluster
    2. That as I flip-flopped between instances, the session value was available all the time.
  • Again, the replication would work and for several refreshes I could see my session variable available on each instance...until it wasn't. After a random number of refreshes/seconds (between 2 - 10 refreshes say) the value would disappear.

I am at a loss to explain why this is happening. We considered using Redis as a session store to see if it helped but frankly, our team has no experience with it, it is clunky to get working in Windows and we really don't want any more moving pieces in our infrastructure if we can help it.

Any insight on what is occurring as well as advice for how to peer behind the scenes as it were and see what is going on with session replication would be greatly appreciated.

Thanks

TOPICS
Server administration

Views

1.7K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Sep 28, 2018 Sep 28, 2018

Copy link to clipboard

Copied

I'm not going to answer your question right now. Instead, I'm going to ask a different question: do you really need session replication? Would using sticky sessions at the load balancer solve your problem? That's a much more reliable solution in my experience. If you can't use sticky sessions - and there are valid reasons why that might be the case - I'll try to respond more fully when I can.

Dave Watts, Fig Leaf Software

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Sep 28, 2018 Sep 28, 2018

Copy link to clipboard

Copied

Hi Dave,

Thanks for your reply. To answer your question: in short, I'm not sure.

We currently have a very simple architecture with one web server and one db server. The architecture we are moving to is new to us and we are basing decisions off of what we have gleaned from various sources.

Bottom line is that we were using session replication to ensure that should something go wrong with any of the CF instances, then a user could be pointed to a working instance and continue using the system uninterrupted.

Basically we are trying to achieve the following:

  • We want to improve performance/response. Distributing user load across multiple instances key here. Still unsure if having multiple instances local to a single server is of any benefit.
  • We want to provide failover capability with little disruption to users. Instance/server could go down without impacting the users.

If I turn on sticky sessions within Coldfusion, then true, I don’t seem to have any problems. However, I had assumed sticky sessions would be problematic  because if the instance/server to which a user is connected goes down, then it would force them to login again when they are inevitably shunted to another instance/server, marring the user experience. Also, once whatever issue with the down instance/server has been resolved, because sessions are sticky, all the traffic that has temporarily been moved to the functioning instance/server will not be redistributed - we'll have all users on one instance/server and none on the other.

You mentioned turning them on within the load balancers so maybe you are not referring to CF sticky sessions at all. I'm not sure how this can be managed at the load balancer but I would love your insights. When I think about it though, I wonder if the same problem wouldn't happen at load balancer level - if not replicated, sessions are lost and once optimal conditions are restored, traffic will still be 'stuck' to the server that wasn't down.

Thanks

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 02, 2018 Oct 02, 2018

Copy link to clipboard

Copied

I think you have a pretty good read of things. You're right, with sticky sessions only and not using session replication, you won't be providing failover. But you will be providing overall redundancy and (hopefully) improving performance and response time. I can't really guarantee even that, because you might have other bottlenecks that are preventing that - how database access works with your application, for example.

My own opinion is that most applications don't really need session replication, because the inconvenience to users is usually pretty minor. If you have a shopping cart application, this might not be the case. Otherwise, though, that's more of a business decision than a technical one.

And yes, I was referring to using sticky sessions at the load balancer. I'd personally much rather use it there than in the peer-to-peer CF session management stuff. There's no real upside to using sticky sessions within CF's own session management.

And, yes, if you bring up the failed server, existing users will still be processed by the other server. In most applications, this isn't a huge problem because they'll naturally drain off after a while. This is especially true if you keep session timeouts low (like you should anyway).

The benefits of session replication are pretty neat, but honestly unless you want to use Redis for it I wouldn't recommend it. The previous version of session replication is a little bit fragile and there isn't really a way to fix it in production if you have a problem. Also, it doesn't scale well beyond two servers. I understand your not wanting to add new pieces of infrastructure, but that's really what scaling out requires. Building a reliable multi-server infrastructure is generally a lot more complicated than building a single-server environment - if you have two servers instead of one, you're going to have more than twice the number of moving parts than you had with one.

Dave Watts, Fig Leaf Software

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 02, 2018 Oct 02, 2018

Copy link to clipboard

Copied

Do you use:

the setting loginStorage="session" on both instances?

the cookiedomain attribute of the cflogin tag?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Oct 05, 2018 Oct 05, 2018

Copy link to clipboard

Copied

Hi BKBK, thanks for you reply.

We are not using cflogin for our login, so that is moot. We also did not explicitly set loginStorage in our application.cfc - I tried as you suggested with no effect.

Thanks

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 05, 2018 Oct 05, 2018

Copy link to clipboard

Copied

emanresu1234  wrote

We are not using cflogin for our login, so that is moot. We also did not explicitly set loginStorage in our application.cfc

How then do you maintain login across requests?

You said earlier:

  • Upon logging in, I was immediately returned to the login screen (app checks for user info in session and redirects to login if not found)
  • Debugging revealed that I was actually being logged in but after redirecting to some new page after login the user info would be gone from session.
  • Multiple login attempts in a row (same credentials, just tried over and over again and again) revealed that sometime slogin would proceed just fine and I would get into the app. However, if I refreshed the page or went to another page, the session would be lost very soon but at random (within a few page refreshes).

In my opinion, the neatest way to achieve this is by using cflogin, combined with loginStorage="session".

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Oct 05, 2018 Oct 05, 2018

Copy link to clipboard

Copied

Hi BKBK,

We use our own login forms and have code to set various user-related session values manually.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 06, 2018 Oct 06, 2018

Copy link to clipboard

Copied

emanresu1234  wrote

Hi BKBK,

We use our own login forms

That goes without saying. The login credentials from the form last for the duration of one request. Hence the need to store them to make them persistent. Some ways to do so include session scope, client scope and database.

So, let us assume the user has submitted the form and you have validated his credentials. How do you persist this information from one request to another throughout the session?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Oct 05, 2018 Oct 05, 2018

Copy link to clipboard

Copied

Hi Dave, thanks for your insights. You're right, there are certain complexities that will just come part and parcel with creating a multi-server architecture.

It is at least helpful to hear that others have had issues with a certain flakiness when attempting to use vanilla session replication without redis. What documentation there is (IMO, there isn't enough) made it seem like using redis is a personal preference rather than a necessity.

It may anger some users from time to time, but I will probably go with just using stick sessions for now and aim for switching to session replication with redis once we have time to get to grips with it.

Thanks.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 05, 2018 Oct 05, 2018

Copy link to clipboard

Copied

I'm happy I could help!

Dave Watts, Fig Leaf Software

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Apr 26, 2022 Apr 26, 2022

Copy link to clipboard

Copied

Hi Guys @Dave Watts @BKBK ,

I am interested on getting session replication working between 3 instances of CF2018 Standard.

Will that work?

My setup is as follows (similar to @emanresu1234 ) but with some important differences.

  • 3 web servers behind load balancers
  • On each web server sits a CF Standard instance
  • Using J2EE session variables
  • Coldfusion session timeout set to 1 hours

Is there an option to have a cluster aware environmet for failover without using CF Enterprise?

(Sticky sessions is not an option for us as we want to have failover capability)

Thanks.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Apr 26, 2022 Apr 26, 2022

Copy link to clipboard

Copied

Ismael, sadly no. The CF session replication being discussed here is indeed offered in CF Enterprise only.

 

But as also was mentioned in passing, since cf2016 one can store CF sessions in a Redis db instead--whether in CF Standard or CF Enterprise.

 

That does require first that you implement Redis somewhere (free if deployed locally, usually, and free or commercial if deployed via SaaS offerings such as Redislabs, AWS, etc). Then you would simply point CF at that. It does also require disabling j2ee sessions--and cf sessions can be made as "secure" as j2ee sessiins. All this is done on the cf admin memory variables page, top and bottom.

 

PS While you directed your question to Dave and BKBK, I'm assuming that's merely because they were the only prior participants in this 2018 thread, and perhaps you'd hope to get their attention if they may not notice your reply otherwise. 


/Charlie (troubleshooter, carehart.org)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Apr 26, 2022 Apr 26, 2022

Copy link to clipboard

Copied

CF Standard doesn't support what Adobe calls "session replication". That's strictly an Enterprise feature. It relies on peer-to-peer replication which presumably was difficult to develop in the first place. Personally, I don't really like it too much. I've found it to be pretty fragile and, surprisingly for an Enterprise-specific feature, it doesn't scale that well for me - probably because of the whole peer-to-peer thing.

 

Instead, you can build your own "session replication" of sorts using databases, etc. The problem with this approach is that it's a lot of extra stuff you have to implement and it'll be slower than session replication. However, it will be more durable than session replication, and will survive disconnections etc where session replication will not. And it'll also scale well, as the number of servers you add will not make it significantly slower.

 

@Charlie Arehart recommended using Redis for storing sessions. I honestly don't know how well that will work if the Redis server isn't on the same machine as CF, so I'd be a little reluctant to spend a lot of time pursuing that without some testing beforehand. But the overall idea is sound. He may also be able to say that it works just fine in practice. That said, I've done a few implementations using a regular old database like SQL Server or MySQL and just writing queries to save or retrieve data as needed.

 

Final thing: one hour is really a long time to keep session variables available, especially if you're replicating them across multiple servers. Session variables are stored in memory, and that's usually the hardest storage location to work with when you have a lot of data. So, you want to reduce the storage time as much as you can without inconveniencing your users. My unasked-for recommendation would be to drop that back down to twenty minutes or less. However, this can tie in nicely if you're only using session variables to store the user's authentication state and similar odds and ends. If you store the user's cart (assuming this is an ecommerce site) in the database, once the user logs in you can fetch the cart. If the user's session times out, you can fetch it again the next time they log in.

 

Dave Watts, Eidolon LLC

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Apr 26, 2022 Apr 26, 2022

Copy link to clipboard

Copied

Well, as for the prospect of redis being on a server separate from cf, the prospect of network overhead would apply as well to sessions stored manually in a db, right? 🙂 And FWIW, redis is a memory-first db (with optional persistence and even replication among its own instances), which could give it an advantage over traditional disk-based databases (buffering in memory being there also, of course).

 

Anyway, yes, I can report from experience seeing no noticeable negative impact of using redis sessions. Of course, everyone's "mileage may vary", with circumstances ranging from architecture to volume to config to coding, and so on.

 

As you so wisely put it, folks should just test things for themselves. But it is the most ready alternative for sharing sessions across cf instances running cf Standard. 


/Charlie (troubleshooter, carehart.org)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Apr 26, 2022 Apr 26, 2022

Copy link to clipboard

Copied

LATEST

"the prospect of network overhead would apply as well to sessions stored manually in a db, right?"

 

Yeah, but I wouldn't do that either if I could help it. I'd just use sessions for storing the minimal state data a user needs to interact with the site: identity, authentication status, etc. Then, I'd use a unique key from the session to interact with MySQL or whatever. I wouldn't want to wait for the session to load from a RDBMS before I could continue.

 

Also, there's a bit of laziness on my part since I've done this before we had Redis to work with, so why should I rebuild the wheel?

 

That said, thanks for letting me know it works!

 

Dave Watts, Eidolon LLC

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Resources
Documentation