I'd make my jQuery call look something along the lines of (to keep it simple, I'm just editing the jQuery example code from the URL in the OP):
$("#myform").validate({
rules: {
email: {
required: true,
email: true,
remote: {
url: "http://mydomain.com/my/path/to/cfcs/mycfc.cfc?method=checkemail&returnformat=json",
type: "post",
data: {
email: '#the_cf_variable_to_check#'
}
}
}
}
}
});
For my CFC, I'd set up something along he lines of:
<cfcomponent>
<cffunction name="checkemail" output="true" returntype="void">
<cfargument name="email" required="true" type="string" />
<!--- set the default value to false --->
<cfset var emails = StructNew() />
<cfset emails.found = false />
<!--- code to check email in system --->
<cfquery name="findemail" datasource="#dsn#">
select email from table where email = <cfqueryparam type="cf_sql_varchar" value="#arguments.email#" />
</cfquery>
<cfif findemail.RecordCount is 1>
<cfset emails.found = true />
</cfif>
<cfoutput>#SerializeJSON(emails.found)#</cfoutput>
</cffunction>
</cfcomponent>
From the docs, it states the plugin is expecting JSON in return. To ensure you get JSON, set the returnformat key/value to json in the calling code (jQuery block, above), ensure that your CFC method does output content (output=true) and use the CF function SerializeJSON to convert your structure to JSON when outputting it from your method.
NOTE: SerializeJSON will convert keys to upper case, so emails.found, when converted to JSON will be emails.FOUND. Initially, I wrote the example CFC to just output the value true or false but changed it to JSON after rereading the plugin docs.
Hope that helps!
Craig