Element urlmap.servicename is undefined in STDSN.

[dejank52055655]

I imported the CF2018 setting using CFadmin package manager into newly installed CF2021 and now I'm getting a following error when I try to edit ORACLE specific Datasource. (MySQL and MS SQL are OK)

Element urlmap.servicename is undefined in STDSN.

I recreated one datasource and it seems to be ok, however, I have over 200 DS's that would prefer not to recreate manually. CF is running on Centos 7.9, cf v2021. Any suggestions?

[Charlie Arehart]

Well, on the surface it sounds like you're saying you have some setting you want to apply to all the DSNs in an automated way. You could use the CF Admin API, which has a CFC and methods for manipulating DSNs, programatically. It's in your instance's wwwroot/CFIDE/adminapi folder, which is under the "cfusion" folder if you have only one instance.

Or someone may point out how all the DSN info is stored in a neo-datasource.xml file, in the lib folder within your instance, such as cfusion/lib. You COULD try to manipulate that by hand, but that can be risky and error-prone. Since you say you have a DSN that DOES work, you could find what differs between the one that DOES and does NOT work, which can be done easily with the right tools. That may readily identify a simple change to be made in the rest. I can help you do that on a consulting basis (perhaps in less than 15 minutes).

As for how it happened, and especially if you need to prevent it happening again, can you clarify a couple of things?

• first, you refer to using the CF2021 Package Manager, which I assume was to pull in support for datasources. Is it that you installed CF2021 using the zip installer (which implements no features by default) rather than the full/gui installer (which installs all packages by default)? I ask just to know how you have implemented CF2021, in case that becomes relevant
• You refer to having over 200 DSNs, but you don't mention how they got there. The implication is that they got imported somehow. How did you do that?
  • Did you install CF2021 on a machine that already had an earlier version? If so, then on first launch of the CF Admin, it would offer to import the settings from the earlier version.
  • Or did you use the CF CAR (ColdFusion Archive) feature?
  • Or did you (or someone else) just copy in the neo-datasource.xml file from the older version? That would be a BAD idea, as it often leads to the kind of problems you are seeing.
  • If you did either of the first two, there will be a log (a migration log or a CAR log) in the CF logs folder. Do you see anything in there indicating a problem?

But again, the last set of questions are just offered to try to help solve the root cause problem. I realize you just wanted to somehow tweak the DSNs manually, and so I started with that. Hope it's helpful, and happy to help even today if you want assistance.

Please do let us know how it goes.

[dejank52055655]

Hi Charlie,

thank for helping trouble this.

The intent was not make any modification to the datasources, the task was just to import data sources from cf2018 to cf2021.

I edited the neo-datasource.xml (I backed up xml file and stopped the cf instance first) and made comparisons between the datasource A that was not working and the one I manually recreated B (That is working). There were minor differences, mainly in element order and datasource A had sendStringParametersAsUnicode=NO where datasource B had sendStringParametersAsUnicode=false,  however, even after I made those minor formatting changes to datasource A, it was giving me exact error again.

• first, you refer to using the CF2021 Package Manager, which I assume was to pull in support for datasources. Is it that you installed CF2021 using the zip installer (which implements no features by default) rather than the full/gui installer (which installs all packages by default)? I ask just to know how you have implemented CF2021, in case that becomes relevant. Correct. I used cf2021 zip installer. 
• You refer to having over 200 DSNs, but you don't mention how they got there. The implication is that they got imported somehow. How did you do that? Using CF CAR feature, I exported (Created archive) of all settings from cf2018 (including datasources) then I deployed the existing archive file (.CAR) to cf2021 successfully. I had to change custom tag paths and mappings, but all settings imported including datasources successfully.
• Did you install CF2021 on a machine that already had an earlier version? If so, then on first launch of the CF Admin, it would offer to import the settings from the earlier version. Yes; However, I did not chose to import settings using this method.
• Or did you use the CF CAR (ColdFusion Archive) feature? Yes; I used CF CAR feature to create archive file, select settings (all), and to deploy this file (.car) to CF2021.
• Or did you (or someone else) just copy in the neo-datasource.xml file from the older version? That would be a BAD idea, as it often leads to the kind of problems you are seeing. No. To my knowledge no one moved/copied  neo file. I'm the only one working on this upgrade. There is a neo-datasource.bak file that was there but I'm assuming CF made a backup.
• If you did either of the first two, there will be a log (a migration log or a CAR log) in the CF logs folder. Do you see anything in there indicating a problem? I check the CAR log file, I see some errors, I'm not sure what exactly they mean, but nothing pointing to datasources
• "Severity","ThreadID","Date","Time","Application","Message"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:35","","/..../coldfusion2021/<instanceA>/logs/car_deploy_002D2018_002Dsettings.log initialized"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:35","","Restoring entry '{server_root_dir}/bin/jvm.config' to file '/..../coldfusion2021/<instanceA>/bin/jvm.config'"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:35","","Restoring entry '{/..../coldfusion2018/<instanceA>/gateway/cfc/examples/menu/main.cfc}' to file '/..../coldfusion2021/<instanceA>/gateway/cfc/examples/menu/main.cfc'"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:35","","Restoring entry '{server_root_dir}/gateway/config/sms-test.cfg' to file '/..../coldfusion2021/<instanceA>/gateway/config/sms-test.cfg'"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:35","","Restoring entry '{WorkingDir}/server_settings.xml' to file '/..../coldfusion2021/<instanceA>/runtime/work/Catalina/localhost/tmp/server_settings.xml'"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:35","","Restoring entry '{WorkingDir}/archive_properties.xml' to file '/..../coldfusion2021/<instanceA>/runtime/work/Catalina/localhost/tmp/archive_properties.xml'"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Restoring Debugging Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Restoring Remote inspection Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Restoring Server Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Restoring Logging Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Restoring Mail Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Restoring Variables"
  "Error","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Error occurred while updating variable settings"
  "Error","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","An error occurred while trying to encrypt or decrypt your input string: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.. "
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:37","","Restoring Caching Settings"
  "Error","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Error occurred while updating JCS DSN name caching settings"
  "Error","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","null"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Charting Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Custom Tag Paths"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Security Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring System Probes settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Watch Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Update Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Websocket Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring PDF Service Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Web Services Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Solr Settings"
  "Error","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Error occurred while updating settings:"
  "Error","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Path /..../coldfusion2018/<instanceA>/jetty/multicore to the Solr Home is not valid. "
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Font Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Directory Mappings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Scheduled Tasks"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Java Applets"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring CFXs"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Solr Collections"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:31:39","","Restoring Datasources"
  "Warning","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Value for Max Pooled Statements is set to 100 for following drivers - DB2, Informix, Oracle, Microsoft SQL Server, MySQL_DD (DataDirect), Sybase."
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Restoring Web Services"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Restoring Rest Services"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Restoring PDF Services"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Restoring Event Gateway Instances"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Restoring Event Gateway Types"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Restoring Event Gateway Global Settings"
  "Information","ajp-nio-0:0:0:0:0:0:0:1-8012-exec-1","03/09/21","13:33:29","","Deploy complete"

[Charlie Arehart]

Well, bummer. But I have a few more thoughts for you:

• In doing the comparison of the neo-datasource.xml files, I will just note that there could be even a slight difference that might be missed. The key is having a good tool that can compare XML, especially that wddx-formatted XML that CF uses for its neo xml files. If you wanted to reach out to me directly (see my signature), and send me the two files, I could confirm your findings (there's no security risk about the passwords, because they are not only encrypted but with a key that only your CF instance would have).
• As for the error, your screenshot shows an XML comment. There should be none in there by default. Are you sure there's not perhaps a problem whereby there could perhaps be a missing closing comment? or someone perhaps mistakenly used CFML comments (with 3 dashes) versus an XML comment (2 dashes), on either end of such a comment?
• Have you tried running the XML through any sort of validation? That may help you spot something else amiss, tha tcould have been missed in eyeballing with a comparison (and if that comparison tool di dnot validate the XML, either)
• If we can find no other explanation, then it would seem you ought to create a ticket at tracker.adobe.com. I just confirmed there are no others related to what you report, for CF2021 (looking either for "oracle" or "stdin")
• But finally, back to your original question and challenge: you asked for a way to automate correcting the 200 DSNs. Did you have in mind what you wanted to do? I shared how the Admin API might help, but I realize now that if the CF Admin can't edit a DSN, the AdminAPI would likely barf on it...but perhaps not, or perhaps it may offer a different error. Just something to think about 

I know it stinks when a problem like this arises...and you don't know what to blame...and you just want the problem solved. I can only offer ideas for you to consider. As always, just trying to help.

[dejank52055655 · Accepted solution]

• In doing the comparison of the neo-datasource.xml files, I will just note that there could be even a slight difference that might be missed. The key is having a good tool that can compare XML, especially that wddx-formatted XML that CF uses for its neo xml files. If you wanted to reach out to me directly (see my signature), and send me the two files, I could confirm your findings (there's no security risk about the passwords, because they are not only encrypted but with a key that only your CF instance would have). Let me see if I can find a tool to do this in more details, if not I'll reach out to you.
  
• As for the error, your screenshot shows an XML comment. There should be none in there by default. Are you sure there's not perhaps a problem whereby there could perhaps be a missing closing comment? or someone perhaps mistakenly used CFML comments (with 3 dashes) versus an XML comment (2 dashes), on either end of such a comment? Good point, I'll check that.
  
• Have you tried running the XML through any sort of validation? That may help you spot something else amiss, tha tcould have been missed in eyeballing with a comparison (and if that comparison tool di dnot validate the XML, either) I did consider that, I found some online, but I was not sure to do it because of all the information the xml file had. I will look for a desktop app and try that.
  
• If we can find no other explanation, then it would seem you ought to create a ticket at tracker.adobe.com. I just confirmed there are no others related to what you report, for CF2021 (looking either for "oracle" or "stdin") I might have to do that in the end.
  
• But finally, back to your original question and challenge: you asked for a way to automate correcting the 200 DSNs. Did you have in mind what you wanted to do? I shared how the Admin API might help, but I realize now that if the CF Admin can't edit a DSN, the AdminAPI would likely barf on it...but perhaps not, or perhaps it may offer a different error. Just something to think about. Just to clarify, sometimes I forgot to include important details, the only reason we noticed this issue is because one of my coworkers wanted to "edit" the datasource to find the schema name, and he got the error, otherwise it would be unoticed until its in production. Sorry for misunderstanding, I did not want to change datasources at all, they should be correct as is, (at least they were in cf2018), sorry if I was not clear in the original post. (But its good to know for future that we can change them through Admin API)

[dejank52055655]

Hi Charlie,

So according to the tools I used Red hat extension in visual code, the xml file is valid.

I tried to submit the bug to the URL you provided, with all the required information and I got the error that bug cannot be submitted without any information as to why. Check attachment.

Any other way to report this as potential bug?

 

[BKBK]

You got the "Bug Submission Failed" probably because your session had expired. Try submitting the bug again.

[dejank52055655]

I forgot to mention that those datasources are in fact working, the apps are using them, however we just cannot edit thos sources. Also when you click on verify datasource (Little check mark) it does not do anything, or at least does not give any confirmation message if the source onnected or not. Also we are running on Linux Centos 7.9, we have 5 stand alone  network load balancd servers, 4 CF instances per server with about 250 DSN per server.

[BKBK]

@dejank52055655 : Element urlmap.servicename is undefined in STDSN.

I do believe this to be a ColdFusion bug. Hence my earlier suggestion that you report it. 

The likely cause is the generation of corrupt neo-datasource.xml scripts. 

[BKBK · Accepted solution]

@dejank52055655 : Element urlmap.servicename is undefined in STDSN.

At first, I thought it was a bug in /CFIDE/administrator/datasources/oracle.cfm. But I can now prove it isn't. 🙂

To reproduce a similar error, simply point your browser to the page http://yourdomain/CFIDE/administrator/datasources/oracle.cfm.

The cause is therefore obvious. The part of your migration script neo-datasource.xml relevant to Oracle datasources has been corrupted. As a result, when you use the script, the ColdFusion Administrator datasource page posts incorrect form data.

In fact, you can see in the picture in your original post that an XML comment tag "<!--" is being posted as part of the Service Name. Such a corrupted form field value eventually causes an error in /CFIDE/administrator/datasources/oracle.cfm.

In short, you have to fix the migration script. Pay attention to the XML elements pertaining to Oracle datasources, to Service Name and to the location of XML comments. It might actually help to delete all the XML comments.

[dejank52055655 · Accepted solution]

Thanks for the response, there is issue with neo-datasource.xml file. One of my collegues suggested we put in a empty servicename node inside 

...

<var name="DATA_SOURCE">
   <struct type="coldfusion.server.ConfigMap">
       <var name="urlmap">
           <struct type="coldfusion.server.ConfigMap"> (Inside this node)

                 <var name="servicename">
                            <string>
                            </string>
                    </var>

....

and this resolved the issue. We just have to use CF code to insert the missing element.

I agree this is a bug (https://tracker.adobe.com/#/view/CF-4211297) I submitted it but not response yet.

[Charlie Arehart · Accepted solution]

Thanks for sharing this,  dejank. One thing is not clear from your answer here (or in the bug report you created and your first comment there offering this workaround), and I want to offer it for future readers. Please confirm if you agree.

Folks will find that for whatever datasource they are looking at, they will have the var name="yourdatasource" and then that first <struct type="coldfusion.server.ConfigMap"> line right inside of in, then at some point (within that) they will find the <var name="urlmap">  you mention (there may be other elements before it), and then inside of that will be yet ANOTHER configmap line (like you show), which may or may not have other elements inside of it, and it's within THAT that they should add your last 4 lines.

Of course, messing with (even just looking at) these neo*.xml files is challenging because they are not only XML (usually just one big long line) but also "wddx-formatted" XML, making it all the more clumsy. It would help to open the XML in any sort of editor or online tool that can render it into a nicely formatted hierarchical display. (In fact, those on Windows will find that using lowly Internet Explorer does a really NICE job of at least DISPLAYING XML file content in such a hierarchical way.)

You do then have to be VERY careful when editing the file, as the slightest mistake in leaving out a bracket or quote could render it unable to be processed by CF. Indeed, before one does edit the file, they should absolutely make a copy in case they mess anything up, so they can revert to where they started.

And you didn't mention it here, but of course one would need to restart CF after making any such change, for it to take effect.

Hope all that's helpful, for others who may find this and jump to your offered solution.

BTW, you (or someone) had kindly marked my one of my earlier replies as one of 3 listed "correct answers", but I have unmarked it, as it may have some value but really yours is THE answer--and BKBK did indeed point you to that indication in the error of an issue related to "servicename". Those both remain marked as "answers".