Entries Tagged as 'ColdFusion'

Introducing validORM: A Development Tool for Creating ORM CFC And ValidateThis Files

AngularJS , CFML , ColdFusion 5 Comments »

In my previous post about relating different data constructs in AngularJS to each other via property names, I mentioned that I figured out that technique while working on an internal development tool.  This blog post is about that tool.

The AngularJS-powered validORM tool lets you design the ID and column properties of your ColdFusion/CFML ORM CFCs while at the same time creating a matching set of validation rules using the ValidateThis (VT) library (a great library for handling both client and server-side validation).

The first view of the tool is a menu view.  You can choose to generate an updated version of your CFC and VT rules file based on a previous session or create a new set of files from scratch.  Either choice takes you to the generator view.

The first two sections of the in the generator view are pretty obvious:  that's where you choose the name and database table for your ORM object and configue the ID property.

The third section, where you define your column properties and create your validation rules, is where AngularJS really comes into play.  The property drop-down selection, the form controls and the hover hints for each property attribute you add, and the validation rule options and parameters:  those are all AngularJS manifestations of JavaScript data constructs.  Using the "Add Attribute" button to add a property attibute simply adds a JavaScript object of that name to the dataset; the creation of the form elements on the page for that attribute is handled automatically by AngularJS based on the Angular directives in the HTML.  That's Angular's strength: driving page behavior purely through data manipulation. 

The data constructs created by your choice of property attributes and validation rules are separate from the JavaScript objects and arrays of objects that represent all of the attribute and rule options, but they reference each other via the attribute and rule key names.

The last two sections allow you to further define any ValidateThis conditions and contexts referenced in your column property rules.  When you're done, clicking the submission button will trigger the file generation process.  Your ORM and VT configuration data will be parsed by a CFC which will then generate three files in the output folder of the tool:

  • An ORM CFC written in script format.
  • A ValidateThis rules definition file matching that CFC written in JSON format.
  • A JSON-formatted snapshot of the configuration data, with a filename reflecting the name of the object and the timestamp when the file was generated.  Said file will then appear in the menu view of the tool.

Some caveats:  the tool only allows for one ID property, doesn't include absolutely every type of property attribute (just the majority of them), and it doesn't let you define ORM relationship properties.  And as my first foray with using AngularJS, I'm sure it violates one or two Angular best practices (if you're an Angular guru, feel free to chime in with any suggestions for improvement).

But it works and can provide a kickstart to creating any ORM files and ValidateThis rules for your projects.  It comes with an example "Employee" configuration set that you can play with and then delete once you start using the tool.

The tool is available for download (and modification) via GitHub:  https://github.com/bcswartz/validORM. I also created a short video of the tool in action and posted it on YouTube at: http://www.youtube.com/watch?v=-fnt_n65NWg

Quick CFML Tips: Script-based Queries And Tag/Script Hybrid CFCs

ColdFusion No Comments »

While refactoring a particular process in a ColdFusion project at work, I learned two things I thought worth passing on:

1.  There are certain scenarios where doing queries in cfscript, instead of using the <cfquery> tag, does come with a significant performance penalty.  In my case, the scenario was one where I was taking two recordsets created by querying the database and running subqueries against those recordsets in a recursive fashion.  My guess is that instantiating the query object needed to perform query functions in cfscript (which I was doing with each recursion) was the main issue.  When I converted the queries and functions involved in the recursive process to use tag-based syntax, the amount of time it took the process to run dropped by over 33%.

Not saying that this means you should avoid doing script-style queries, just that if there are a large number of such queries in a long-running process, you might want to experiment with refactoring them.

2.  I already knew that I could have one or more cfscript blocks inside the body of a CFC function, but I didn't know that I could combine tag-syntax functions and script-syntax functions in the same CFC like so:

<cfcomponent accessors="true">
    <cffunction name="doX" output="false" returntype="void">>
        ...
    </cffunction>

    <cfscript>
        public void function doY () {
            ....
        }
    </cfscript>
</cfcomponent>

...it had never crossed my mind to try it that way. Certainly useful when you need to run a CFML function that has no cfscript equivalent in whatever version of CFML engine you're using (I should note that I was using ColdFusion 9.0.1 in both of the above cases).

Adding Oracle Support to BugLogHQ: How the Application Design Made it Easy

ColdFusion 3 Comments »

As I mentioned in my previous blog post, in order to try out BugLogHQ in my work environment, I knew I would have to update the code so it could run against Oracle database tables.  So I started looking through the code to see how difficult of a task that would be, to see if it was worth the trouble.  Changing the code to support another database would mean at least looking at (if not updating) every block of code that performed a SQL operation:  if there were a lot of such blocks, it could be a time-consuming task.

Fortunately for me, BugLogHQ was designed to abtract most of the SQL operations behind a DAO layer.  Each database table used by BugLogHQ is mapped to a DAO CFC in the components/db folder of the BugLogHQ application, and those CFCs spell out the name and cfqueryparam data type for each table field and inherit getter and setter-type functions from a base DAO class.  When it comes time to persist data from instantiated DAO objects into the database, the DAOs are processed by the dbDataProvider.cfc in the components/lib/dao/ folder, and that CFC contains the SQL code for performing all of the CRUD operations.

(I should point out that this is a somewhat simplified description of all of the "moving parts" involved in data persistence in BugLogHQ, but after following the code through the process of adding and removing records I determined that I didn't need to make changes to other objects such as the DAO factory).

The main issue I needed to address was the fact that the current code for inserting a new record took advantage of the autoincrement feature in the other databases supported by BugLogHQ (MySQL, MS SQL, etc.).  Oracle doesn't come with an autoincrement field option.  Instead, Oracle has what are called sequences, which are essentially autoincrementing numbers that exist separately from the database tables (meaning you could, if you wanted to, pull the next integer from a particular sequence into any table).  So in the SQL install script I wrote to create the needed tables in Oracle (mirroring the install scripts for the other database engines present in the BugLogHQ install folder), I added lines to create sequences for each table, with each sequence name being the table name with "_seq" appended to the end of it.  I then added code to the _insert() function in the dbDataProvider.cfc to use the sequences for the primary key value when inserting a record:

  
<cffunction name="_insert" access="private" returntype="any">
  <cfargument name="columns" required="true" type="struct">
  <cfargument name="_mapTableInfo" type="struct" required="true">
  ...  
  <cfset var dbtype = variables.oConfigBean.getDBType()>
  <cfset var lstFields = structKeyList(arguments.columns)>
  <cfset var tableName = arguments._mapTableInfo.tableName>
  <cfset var seqName= tableName & "_seq">
  <cfset var pkeyName= arguments._mapTableInfo.PKName>
  ...
  <cfif dbtype eq "oracle">
    <cftransaction>	
      <cfquery name="qry" datasource="#DSN#" username="#username#" password="#password#">
        INSERT INTO #getSafeTableName(tableName)# (#pkeyName#,#lstFields#)
        VALUES (
        #seqName#.nextVal,
        <cfloop list="#lstFields#" index="col">
          <cfqueryparam cfsqltype="#arguments.columns[col].cfsqltype#" value="#arguments.columns[col].value#" null="#arguments.columns[col].isNull#">
          <cfif i neq listLen(lstFields)>,</cfif>
            <cfset i = i + 1>
          </cfloop>
        )
      </cfquery>
				
      <cfquery name="qry" datasource="#DSN#" username="#username#" password="#password#" result="qryInfo">
        select #seqName#.currVal as lastId from dual
      </cfquery>
    </cftransaction>
    <cfset newID = qry.lastID>
  <cfelse>
  ...

So if the dbtype value (which comes from the config/buglog-config.xml.cfm file) is "oracle", the Oracle insert code block will execute. The first cfquery block will insert the record, using the nextVal() function of the sequence associated with the table to get the next value of the sequence, while the second cfquery block retrieves the current/latest value of the sequence via Oracle's DUAL "table" (a construct designed to perform pretty much any utility operation that returns a single record or value).

The other issue I needed to address was adjusting some of the field data types.  Oracle's "varchar2" data type can only hold 4,000 characters of text; to store larger amounts of text, you need to use fields with the CLOB data type.  And in order to use cfqueryparam with such fields, you need to set the cfsqltype parameter of cfqueryparam to "cf_sql_clob".  So I updated the code in the entryDAO and extensionDAO CFCs to handle that (with the entryDAO.cfc code shown below):

<cfif variables.oDataProvider.getConfig().getDBType() EQ "oracle">
  <cfset addColumn("exceptionDetails", "cf_sql_clob")>
  <cfset addColumn("HTMLReport", "cf_sql_clob")>
<cfelse>
  <cfset addColumn("exceptionDetails", "cf_sql_varchar")>
  <cfset addColumn("HTMLReport", "cf_sql_varchar")>
</cfif>

After that, all that I had left to do was to add Oracle-specific conditional blocks to two other queries (ones that involved using database functions specific to each database platform) in two other files (components/hq/appService.cfc and components/lib/entryFinder.cfc), and I was done.

Had I done all of the changes I just described perfectly on the first try - which admittedly I didn't  :) - it would have taken me less time than it did to write this blog post.  And that's a mark of tight, well-designed application architecture, where certain key aspects of application behavior are centralized in a manner that lets you add functionality (in this case database engine support) by updating just a few files responsible for handling that particular task.

Need A CF App for Tracking Errors From Multiple Applications? Check Out BugLogHQ

CFML , ColdFusion 5 Comments »

Last month, during a brief lull in my work projects, I decided to tackle an issue I'd been putting off for awhile:  finding a better way of managing errors caught and reported by my ColdFusion applications.  All of my apps are designed to email me a full report of any errors that reach the global error handler, and while that means I'm immediately notified if there's a problem I need to address, the error message folder in my email account ends up serving as a de-facto error archive.  And now that we're (slowly) moving towards building applications as a team, it makes sense to have the errors stored in a centralized location.

After doing a bit of research, I ended up at the home page for the open-source bug logger BugLogHQ, created by Oscar Arevalo.  Based on the project changelog list, it sounded like it might have the features I wanted, but there wasn't a lot of explanatory documentation on the site and the screenshot images were thumbnails that couldn't be enlarged.  And it didn't support Oracle, which is our primary database engine at work (and a perusal of the Google Groups forum for BugLogHQ confirmed that).

Despite those factors, I decided it was worth trying to add Oracle support to the codebase so I could run it and try it out.  I'm glad I did:  BugLogHQ is a well-designed, robust application for managing errors in a way that will still let me keep track of what's happening with my apps.  Now that I've seen how it operates and what it can do, I'm kind of surprised that I had to do research to come across it:  either it completely slipped under my radar or it needs more publicity/exposure in the community.

Screenshot of the dashboard

Several things I like about BugLogHQ:

  • I like the fact that it's easy to get up and running.  Download the files, put them in a "buglog" directory under the webroot (or use a mapping), run the SQL install script for your database of choice (mySQL, PostgreSQL, MS SQL 2000/2005, MS Access, and now Oracle) update the config file with your datasource name and database engine, and simply point your web browser to that "buglog" directory.  After logging in (which forces you to change the default password for the built-in admin user account), you can verify everything's working by going into the settings area of the app and running the tests, which all mimic the methods your other applications (referred to in BugLog parlance as "client" apps) can use to transmit errors to BugLogHQ.  Simple and straightforward.

  • Just like most email clients have rules for filtering and acting on incoming email messages, BugLogHQ provides some rules for managing the errors that come in.  You can set up multiple "mailAlert" rules that will forward on particular errors (filtered by error type/severity, the source app's hostname or application name, or keywords in the configured error message text) to an email address of your choosing, or you can create a mailRelay rule that relays every error received to an email address.  And the emails sent out by BugLogHQ aren't just notifications about the errors:  they provide the full error details so you don't have to log into BugLogHQ to get more information.  There are also rules for discarding certain errors or for monitoring "heartbeat" transmissions from an app, so if the app is offline for a certain length of time BugLogHQ can report the problem.

  • The methodology BugLog uses to report errors is very well-designed.  If your client app is also a CFML app, you simply instantiate a single CFC (with settings for interacting with the BugLogHQ web service you want to utilize) in either the application scope or a bean factory, then call it with its notification function whenever you want to send information to the BugLogHQ app.  If the client app is reporting an error, you can send BugLogHQ a line of message text, the error struct (the raw "cfcatch" struct), the type/severity of the error, and a struct of any additional information you want as part of the report (like the affected user's ID or name). Here's how I called it from the main.error() controller function in the FW/1 application I used during my testing (where I've instantiated the CFC as a service object):

    public void function error(rc) {
        var extraInfo= {username= session.user.getUsername()};
        variables.bugLogService.notifyService("Error report from TestApp1",request.exception,extraInfo,"INFO");
    }
    

    All of those parameters are optional, so you don't need an error struct if you're reporting something that's a status or informational message rather than an error.  If there's a problem with sending the report to BugLogHQ, it'll fallback to sending an email containing all that information to whatever address you specify so the error doesn't get lost.

  • I also appreciate how BugLogHQ "eats its own dogfood".  If an error occurs in BugLogHQ itself, it'll record the error just like it would for any client app, and if the error occurs during the recording process it'll report the error via email.

  • BugLogHQ utilizes a scheduled task (with a default interval of 2 minutes) referred to in the app as the "BugLog Listener Service" to denote and process incoming error messages.  When the task fires, the error messages that have been received since the last run of the task are processed and then written to the error database.  If there's a problem with writing to the database (which I understandably encountered as I was tweaking the code to support Oracle), it'll maintain the errors in the queue until the next iteration (and you can view the contents of the queue within the BugLogHQ dashboard).  Sometimes you need to restart that service in order for certain changes to take effect. If you somehow forget to reactivate the service, it'll be restarted when it receives a new error report from a client app.

  • If one or more of your clients apps are NOT CFML-based web applications, there are library files provided that let you submit errors from PHP or Python apps, or straight from Javascript.  I haven't tried out those files yet, but we do have a few PHP apps in our shop now that could make use of the PHP option.

  • There are configuration options for tying BugLogHQ to a JIRA instance, providing error summaries via RSS or a digest email, and setting an API key to ensure that only your apps can submit data to BugLogHQ.

Screenshot of full error report

Bottom line:  if you don't yet have an error-logging application or other mechanism for storing and managing errors from your application(s), you should take a look at BugLogHQ.  Oscar and his contributors have done a great job with this application.

Quick ColdFusion Tip: Use ClearParams() When Doing SQL Inserts in a Loop

ColdFusion 3 Comments »

Today I had the need to write code that would loop over a recordset, parse the data in each iteration, and then insert the transformed data into another table.  Nothing I haven't done countless times, but this time I was using the query functions in cfscript to do my inserts.  I knew the resulting code was going to be a bit slow (as inserting thousands of rows takes time), but when I ran it it was a LOT slower than I expected.

After adding some code to time the various parts of my routine, I discovered that the time it took for each insert transaction was steadily growing with each iteration of the loop, to the point where it was taking 500+ milliseconds per insert.  But why?

Then I saw the problem.  I had forgotten to invoke the query object's clearParams() function at either the beginning or end of my loop.  Apparently ColdFusion will let you create a query parameter with the same name attribute using addParam() - as was happening in my loop - and not throw an error (which is what I would have expected to happen), but it leads to a performance issue with the SQL execution.

In the few times where I've reused a query object with different parameters, I've been careful to use clearParams(), but I simply overlooked it this time.  Lesson learned.