Coldfusion Log JSON Format

[Souvik Saha Choudhury]

I need all my Coldfusion logs to store in JSON format.

I can then parse those logs as a single line rather than a multi-line.

How can I achieve that?

[Charlie Arehart]

To be clear, there's no cf feature to enable that. You should be able to find generic 3rd party tools offering to convert a text log file to json.

You may even be able to find some tomcat-specific feature or add-on providing for this.

Either way (and to your last point), do note that there are 2 kinds of cf logs: those that are single-line (most of them), having a date/time at the start of each line. Then there are a few that are multi-line (having lines without a datetime), including the coldfusion-out.log, coldfusion-error.log, and exception.log, though SOMETIMES they are single-line, depending on the circumstance.

If the latter may be what you refer to, I suspect you'll have a heck of a time converting those to single-line, though I suppose with some tools it may be possible.

Let us know if you find anything, or perhaps someone else will have an idea for you. 

[BKBK]

Even if ColdFusion had the facility to do that, why would you bother at all? Just let ColdFusion do its thing. You do yours. 🙂

Converting a ColdFusion log file to JSON format is straightforward. Take the gateway log file, for example:

<cfset logFileAsIs=fileread("C:\ColdFusion2021\cfusion\logs\eventgateway.log")>

<cfset logFileAsArray=[]>

<!--- Using line-break as delimiter, convert each line of log
into an element in an array --->
<cfloop list="#logFileAsIs#" delimiters="#chr(10)&chr(13)#" index="line">
	<cfset arrayAppend(logFileAsArray, "{#line#}")>
</cfloop>

<cfset logFileAsJSON=serializeJSON(logFileAsArray)>

<!---<cfdump var="#logFileAsArray#" label="logFileAsArray">--->
<!---<cfdump var="#logFileAsJSON#" label="logFileAsJSON">--->

[Souvik Saha Choudhury]

@BKBK Thank you for the reply.

I think your solution will only work for single-line log files, not for multi-line log files.

I need to parse multi-line log files also.

Let me know your thoughts.

[BKBK]

Sorry, I don't understand what you mean by "single-line" and "multi-line".

Could you please provide examples to illustrate.

Thanks.

[Charlie Arehart]

Alfred, I know you're asking Souvik, but I did already address this earlier here : "do note that there are 2 kinds of cf logs: those that are single-line (most of them), having a date/time at the start of each line. Then there are a few that are multi-line (having lines without a datetime), including the coldfusion-out.log, coldfusion-error.log, and exception.log, though SOMETIMES they are single-line, depending on the circumstance."

[BKBK]

I think your solution will only work for single-line log files, not for multi-line log files.

I need to parse multi-line log files also.

Let me know your thoughts.

By Souvik Saha Choudhury

Hi @Souvik Saha Choudhury ,

The following code will covert the usual ColdFusion files (in /cfusion/logs/) to JSON. It takes into account that multiple lines of log may occur under a given date-time.

I should like to hear if it helped.

<!--- 
Convert the usual log files in \cfusion\logs to JSON format.
The JSON consists of key-value pairs, the keys being the line numbers.
--->

<cfdump var="#convertLogFileToJSON('C:\ColdFusion2021\cfusion\logs\application.log')#">

<!---
<cfdump var="#convertLogFileToJSON('C:\ColdFusion2021\cfusion\logs\coldfusion-error.log')#">
<cfdump var="#convertLogFileToJSON('C:\ColdFusion2021\cfusion\logs\exception.log')#">
<cfdump var="#convertLogFileToJSON('C:\ColdFusion2021\cfusion\logs\eventgateway.log')#">
<cfdump var="#convertLogFileToJSON('C:\ColdFusion2021\cfusion\logs\server.log')#">
--->


<cffunction name="convertLogFileToJSON" returnformat="JSON"  output="false">
	<cfargument name="logFilePath" type="string" required="yes">
	
	<cfset var logFileContentAsIs=fileread(arguments.logFilePath)>
	<cfset var filename=lCase(listFirst(listLast(arguments.logFilePath,"/\"),"."))>
	
	<cfset var logFileContentAsArray=[]>
	<cfset var lineTriggersArrayEntry=false>
	<cfset var index=1>
	<cfset var logCategory=0>
	
	<cfset var tempStruct={}>
	
	<cfswitch expression="#filename#" >		
		<!--- 
		Case: CF log file in which the first line characters up to and including ' AM ' or ' PM ' 
		form a date-time.  
		For example, coldfusion-error,coldfusion-out
		--->
		<cfcase value="coldfusion-error,coldfusion-out"  delimiters=",">
			<cfset logCategory=1>
		</cfcase>
		
		<!--- 
		Default Case: CF log file in which the 3rd and 4th comma-delimited columns form a date-time.
		For example, application,audit,cfpm-audit,eventgateway,exception,http,mail,monitor,restservice,scheduler,
		server,webservice
		 --->
		<cfdefaultcase>
			<cfset logCategory=2>
		</cfdefaultcase>
	</cfswitch>
		
	
	<!--- 
	1. Use line-break as delimiter to pick out each line of log.
	2. Each log entry is determined by a date-time. So group as one 
	   entry all the lines of log that occur on the same date, 
	   and store them as one element in an array. 
	--->
		
	<cfloop list="#logFileContentAsIs#" delimiters="#chr(10)&chr(13)#" index="line">
					
		<cfset lineTriggersArrayEntry=triggerArrayEntry(logCategory, line)>
		
		<!---Start by handlng the first line of log, which is usually the header.--->
		<cfif index eq 1>
			<cfset logFileContentAsArray[index]={"line1":"#line#"}>
		</cfif>
		
		<cfif lineTriggersArrayEntry>
			<cfset index=index+1>
			<cfset logFileContentAsArray[index]={"line#index#":"#line#"}>
		<cfelse>
			<cfset var tempStruct=duplicate(logFileContentAsArray[index])>
			<cfset structUpdate(tempStruct,"line#index#",tempStruct["line#index#"] & chr(13) & chr(10) & line)>
			<cfset logFileContentAsArray[index]=tempStruct>
		</cfif>
				
	</cfloop>
	
	<cfset var logFileContentAsJSON=serializeJSON(logFileContentAsArray)>
	
	<cfreturn logFileContentAsJSON>
 </cffunction>
 
 <!--- 
 Given a line of log of a particular category,
 this function extracts a substring from line 
 at a location where we would expect a date.
 
 True is returned if the extracted substring is a valid date.
 False is returned otherwise
  --->
<cffunction name="triggerArrayEntry" returntype="boolean">
	<cfargument name="logCategory" type="numeric" required="yes">
	<cfargument name="line" type="string" required="yes">
	
	<cfset var arrayEntryTrigger=false>
	<cfset var testSubstringFromLine="">
	
	<cfif arguments.logCategory eq 1>
		<cfset arguments.line=trim(arguments.line)>
		<!--- 
		Assuming the first characters in the line, up to and including ' AM ' or ' PM ' 
		form a date-time. 
		(the 3 consists of the number of characters in ' AM' or ' PM')
		--->
		<cfset testSubstringFromLine=left(arguments.line, 3+reFindNoCase("^*.\sAM\s|^*.\sPM\s",line,1))>
	<cfelseif arguments.logCategory eq 2>
		<!--- Assuming line to be list with comma as delimiter, extract and merge 3rd and 4th elements --->
		<cfif listLen(arguments.line) gte 4>
			<cfset testSubstringFromLine=toString(replace(listGetAt(arguments.line,3) & " " & listGetAt(arguments.line,4), '"', '', "all"))>
		</cfif>		
	</cfif>
	
	<cfreturn isDate(testSubstringFromLine)>
	
</cffunction>