Skip to main content
August 21, 2007
Question

CF8/JRun4 Cluster for Load Balancing

  • August 21, 2007
  • 5 replies
  • 1852 views
Does anyone have an example of how to set up a CF8/JRun4 cluster for load balancing?

I have three servers:
x004 - Linux - Apache2 (10.0.0.54,10.1.0.54)
x020 - Linux - JRun4/CF8 (10.0.0.70,10.1.0.70)
x021 - Linux - JRun4/CF8 (10.0.0.71,10.1.0.71)

Every server in our network has two network cards. One network card is attached to 10.0.x.x which has a gateway to the internet and runs at 100Mbps and is firewalled, and the other is attached to 10.1.x.x which runs at 1Gbps and is internal with no gateway. I'm trying to set it up so web traffic arrives on 10.0.0.54 into Apache and mod_jrun20 bootstraps a cluster named STST using 10.1.0.54 which consists of STST_x020 coldfusion server running on x020 and STST_x021 running on x021. I want the communications between JRun4 on x020 and x021 to occur on the 10.1.x.x network and eventhough JRun and ColdFusion will only use the 10.1.x.x network I still need the 10.0.x.x network card attached for other purposes which require a gateway. I have installed JRun4/CF8 about 10 times already and it seems I have no control over what network JRun4 clusters on... sometimes it will communicate on one, sometimes the other and without being able to set which network is being used there always seems to be "network error" on at least one of the two CF8 servers. I was able to get everything working fine by disabling the network cards on the 10.0.x.x network and re-installing everything... but as soon as I added the network cards back the whole thing was broken again.

How is this supposed to work? Most of the examples are either no clustering or clustering on the same machine with Apache running on the same box... I don't see any clustering across machines examples.

How do I install a connector on a web server which doesn't have JRun on it and get wsconfig to connect to a multi-machine cluster when wsconfig only accepts a single IP address as a host and the cluster is not listed?

How do I get JRun to bind to a specific network card?

Does this work if I choose a J2EE server other than JRun?

Any help anyone can provide is greatly appreciated. I'm getting close to giving up which means staying on the non-clustered environment and figuring out how we can deal with scalability by switching to something else.
    This topic has been closed for replies.

    5 replies

    Inspiring
    January 13, 2008
    DCwebGuy wrote:
    > Only problem I ran into is that when I installed CF8 Enterprise I told it to
    > use Apache up front. So now that I want to point Apache to my cluster, it told
    > me Apache was already configured for Jrun. So I removed the default "cfusion"
    > Apache instance and created a new instance pointing to my cluster. But now I
    > can't see my Enterprise Manager cluster admin since I guess that only runs
    > under "cfusion". Now I'm left with figuring out how to install another
    > instance of Apache that does not use the same Apache root, or completely
    > reinstall CF and tell it to use its own built-in server initially.

    Uncomment the part about the webserver in jrun.xml and restart the instance.


    > What's the best way to go if trying to developer a very high availability
    > system?

    Build it on your workstation first :)

    Jochem


    --
    Jochem van Dieten
    Adobe Community Expert for ColdFusion
    Inspiring
    January 13, 2008
    From some help in a thread I started here I learned that in JRun4, the JRun admin lives on port 8000, and the "base" cfusion instance lives on port 8300.

    So I decided to copy my original CFIDE directory to the "cfusion" server root under /opt/jrun4/servers then pointed my browser to http://[localhost]:8300 and I was able to see the CF Admin with my cluster manager. Btw, the "cfusion" root was browseable but did not contain any files whatsoever, so that's where I got the idea to just copy the CFIDE there. I don't know if this was right (or smart) but it worked for me.

    I will assume that this is being served through the built-in Jrun web server and not Apache. If I am wrong, or if you can tell me how I actually confirm which web server is serving it, please tell me!

    Thanks. These threads have been very useful.
    Inspiring
    January 13, 2008
    UPDATE: I figured out the 10.*.*.* addresses are my internal IPs on Amazon EC2. Since they are not on the same subnet (as far as Jrun is concerned) I added the internal IPs to that security.properties file and voila, they all connected to my cluster.

    Only problem I ran into is that when I installed CF8 Enterprise I told it to use Apache up front. So now that I want to point Apache to my cluster, it told me Apache was already configured for Jrun. So I removed the default "cfusion" Apache instance and created a new instance pointing to my cluster. But now I can't see my Enterprise Manager cluster admin since I guess that only runs under "cfusion". Now I'm left with figuring out how to install another instance of Apache that does not use the same Apache root, or completely reinstall CF and tell it to use its own built-in server initially.

    What's the best way to go if trying to developer a very high availability system? Thanks.
    Inspiring
    January 10, 2008
    I would like to jump in here because I may be having similar trouble. My cluster test is on Amazon EC2 where I have spun up three servers of the exact same installation...Ubuntu 7.10 and CF8 Enterprise.

    One of the mysteries I have is why the new instances I create resolve to a 10.*.*.* address when that addy is nowhere in the network, unless I'm missing something, which could be true since I am totally new to CF cluster.

    Second, where does Apache fit into this? I opened up a port on Apache for the new instance, restarted it, but still the instance will not connect to itself. Regardless, when I try to set up a remote instance I get the same "network connection' error message.

    I know the 3 machines can talk to each other because I was able to set up multi-server monitoring, and it saw all the other servers. But I don't know how the network card issue comes into play in the compute cloud.

    The Amazon DNS for each server can be resolved to a static IP (although it is not guaranteed long term), but even why I try to use the static IP instead of the DNS I get the same errors.

    Now, since these are all Amazon virtual servers, I assume they are on the "same network". Could I be wrong here? Is Jini smart enough to understand the compute cloud architecture?

    Please advise. It is key for me to be able to use CF8 Enterprise clustering for load balance and fail over in the Amazon EC2 cloud.

    I am even happy to give you direct URL access and password to view my instances. Thanks.
    September 12, 2007
    Hi EnviroTO,

    You mention a technote, but not which one. Try the following one, title 'How to enable JRun 4.0 clustering on machines with multiple NIC cards'. Hopefully that may deal with your multiple nic cards. It mentions the turning off automatic discovery of clustered members and explicitly stating which are in the cluster.

    http://www.adobe.com/go/1e8e9170


    NOT TRIED WARNING ;)
    In terms of the Apache being on a different box than JRun, can you not just put the mod_jrun22.so and make the 'JRunConfig Bootstrap ' and '' settings as appropriate?
    September 12, 2007
    Thanks for the link on the tech note. I will try that out.

    As far as the Apache on a different server than the JRun servers... I have been able to get the connector working but running wsconfig does not discover the multi-server cluster the way documentation suggests it would. If I enter a physical server address it lists the servers running on that one machine but does not show the configured cluster as an option. I have found editing config files manually is more reliable than the wsconfig program anyways, but I don't have a model of the config files for clustering to go by and nothing is being written into jrunserver.store when I pick a server that is in a cluster.
    Participating Frequently
    September 14, 2007
    The article at http://www.adobe.com/go/1e8e9170 is specific to configuring two or more cluster nodes that reside on separate networks, e.g. 10.0.1.0/24 and 10.0.2.0/24. (The article doesn't state it, but you can only use unicast peers if your cluster nodes host a single instance of JRun or multiple instances of JRun in the same cluster domain. When performing unicast discovery, JRun looks for all Jini groups and not just the cluster group.)

    Anyhow, that's not your problem. The simplest solution is you haven't enabled the jrun.servlet.jrpp.JRunProxyService service. I'm most familiar with the Windows version of JRun, but I'm assuming the directory structure is similar across platforms. In <jrun_root>/servers/<name>/SERVER-INF/jrun.xml, set the deactivated attribute of the jrun.servlet.jrpp.JRunProxyService service to false and restart JRun. You should now see JRun listening on the appropriate port. (The default for the first manually created instance is 51000.) You can limit the proxy service to a single interface using the interface attribute.

    If you have enabled the proxy service, verify your security settings in <jrun_root>/lib/security.properties. It's usually best to limit access to specific hosts. Comment out the jrun.subnet.restriction parameter and set the jrun.trusted.hosts to the IP address of your web server, e.g. 10.1.0.54.

    Forcing all JRun processes/services to listen on a single interface isn't difficult, but it does require modifying quite a few configuration files by hand. If you need assistance with that, I can elaborate.

    Configuring the JRun module under Apache is pretty straightforward. If you're not using virtual hosts, it's very simple. If you are using virtual hosts, it's still simple, but your JRun configuration can be virtual host-specific.

    On your Apache server, you'll want to create a directory structure for the JRun module. I'll assume /opt/jrun/lib/wsconfig/1, but you can use anything you want. Once the directory structure is created, extract the appropriate JRun module from wsconfig.jar to the new directory. You're most likely interested in the Apache 2.0 module, wsconfig.jar/connectors/apache/intel-linux/prebuilt/mod_jrun20.so. Let's assume you've extracted the module to /opt/jrun/lib/wsconfig/1/mod_jrun20.so. Your Apache service account should have read, write, and execute permissions on the /opt/jrun/lib/wsconfig/1 directory.

    The JRun module configuration is normally appended to your current httpd.conf file by wsconfig. Here's a sample configuration:

    LoadModule jrun_module "/opt/jrun/lib/wsconfig/1/mod_jrun20.so"

    <IfModule mod_jrun20.c>
    JRunConfig Verbose false
    JRunConfig Apialloc false
    JRunConfig Ssl false
    JRunConfig Ignoresuffixmap false
    JRunConfig Serverstore "/opt/jrun/lib/wsconfig/1/jrunserver.store"
    JRunConfig Bootstrap 10.1.0.70:51000
    #JRunConfig Errorurl <optionally redirect to this URL on errors>
    #JRunConfig ProxyRetryInterval 600
    #JRunConfig ConnectTimeout 30
    #JRunConfig RecvTimeout 30
    #JRunConfig SendTimeout 30

    AddHandler jrun-handler .jsp .jws .cfm .cfml .cfc .cfr .cfswf
    </IfModule>

    You may also want to update your DirectoryIndex directive with an appropriate index page, e.g. index.cfm.

    After the first request to a page handled by the JRun module is received, the module will query the boostrap server, 10.1.0.70:51000, for a list of cluster peers. If you've configured your cluster correctly, a line similar to following will be written to /opt/jrun/lib/wsconfig/1/jrunserver.store:

    proxyservers=10.1.0.70:51000;10.1.0.71:51000

    You can create/edit this file manually as well. Unfortunately, the bootstrap option only accepts one server. If your bootstrap server is down, the JRun module will use the values in jrunserver.store directly, if the file exists.

    Here's a complete list of JRun module options:

    metrics *
    debugger *
    ssl *
    verbose
    traceflags
    serverstore
    bootstrap
    errorurl
    apialloc
    ignoresuffixmap
    proxyretryinterval
    connecttimeout
    recvtimeout
    sendtimeout
    sslcalist

    Options flagged with an asterisk can only be configured at the Apache server level. All other options can be configured at the server level and/or the virtual host level. The usage of these options is in the JRun documentation, and the JRun module source code is included in wsconfig.jar. Keep in mind that versions of the JRun module shipped prior to ColdFusion 8 were coded to assign the connecttimeout and sendtimeout options to the socket connection timeout. Whichever option appeared last in your configuration ended up as the final value. This has been fixed in ColdFusion 8 and presumably the next release of the JRun updater.

    I think that's a good start. If you need more information or can't find what you need in the JRun or ColdFusion documentation, let me know.

    If you're looking for resiliency, I highly recommend expanding your configuration to include a second web server and a hardware load-balancer (preferably one that supports redudancy via multiple paths and devices, e.g. devices from Cisco, F5, or Foundry Networks). Often, however, running Apache on the ColdFusion server(s) provides adequate performance, and round-robin DNS records coupled with the ability to update DNS quickly in the event of a failure may be all you need for load-balancing and failover.
    September 11, 2007
    OK. Nobody clustering out there? How about a hint of what should be in the "JRunConfig Bootstrap" parameter on Apache 2.2 when there is a cluster and what should be in wsconfig.properties (under {jrun.home}/lib/wsconfig/wsconfig.properties.

    I installed the multiserver version, I have the web server able to connect to one or the other but can't figure out how this cluster is supposed to get set up. Any example of the "JRunConfig Bootstrap" parameter and a wsconfig.properties file for a multiserver cluster would be very helpful.