Skip to main content
February 13, 2009
Question

Parse Text File

  • February 13, 2009
  • 7 replies
  • 1694 views
Hi,

I need parse text file and insert to DB.

I am confused...could not parse it.

Text file looks like tab delimited.

How to parse?..please help me.
    This topic has been closed for replies.

    7 replies

    Participant
    May 28, 2009

    Amm85:

    Here is a sample text file "data.txt" that contains data with 3 columns - product id, product qty and product price. Note, my example includes empty rows and empty columns.

    345<tab>7<tab>29.75

    123<tab><tab>30.90

    Here is the script that will generate the sql statement to insert this data into table "product". The sql statement is created in variable $sql.

    var str sql ; set $sql = "INSERT INTO product VALUES "

    # Read file in.

    var str data ; cat "data.txt" > $data

    # We will add a comma before each row except the first row.

    var  bool firstRow ; set $firstRow = true

    # Get one line at a time.

    while ($data <> "")

    do

        var str row ; lex -e "1" $data > $row

        # Convert all tabs to commas.

        while ( { sen "^\t^" $row } > 0 )

            sal "^\t^" "," $row > null

        # Add this row to sql.

        if ( $firstRow )

            # We will NOT ADD a comma before this row.

            set $sql = $sql + "(" + $row + ")"

        else

            # We WILL ADD a comma before this row.

            set $sql = $sql + ", (" + $row + ")"

        endif

        set $firstRow = false

    done

    echo $sql

    The above is in biterscripting ( http://www.biterscripting.com ) .

    Richard

    Inspiring
    February 13, 2009
    > so how will i handle empty values?

    Most CF List functions do not recognize empty elements. With CF8 you can convert the list to an array, then use array notation. Unlike most list functions, CF8's listToArray function will respect empty values if you use the "preserveEmptyElements" attribute.

    http://livedocs.adobe.com/coldfusion/8/functions_l_21.html

    But, since you are using ms sql you should also consider BULK INSERT which is faster than looping. If you still prefer to use looping, you should use cfqueryparam for the values.
    February 13, 2009
    Thanks.

    No empty values in the file now.

    i need to automate this...maybe in future it may happen.

    so how will i handle empty values?
    Inspiring
    February 13, 2009
    > #listgetat(prod_data_line,1,'char(9)')#

    You are not using the functions correctly. The function name is chr(), not ch ar(). Also, remove the single quotes. Otherwise you are saying the delimiter is the literal string 'char(9)', not the tab character.

    http://livedocs.adobe.com/coldfusion/8/htmldocs/help.html?content=functions_c-d_04.html

    #listgetat(prod_data_line, 1, chr(9) )#

    Can the columns in the file contain empty values?

    Inspiring
    February 13, 2009
    > How can i parse tab delimited text file?

    The same way you would a csv file. The difference is you use a tab character (ie chr(9) ) instead a of comma "," to split the column values. I cannot tell you more than that as you did not post the actual code or error message.


    > I am inserting to sql server.

    You may find BULK INSERT a faster method
    http://msdn.microsoft.com/en-us/library/ms188365.aspx
    http://www.sqlteam.com/article/using-bulk-insert-to-load-a-text-file
    February 13, 2009
    Please see code here
    February 13, 2009
    Thanks for your help.

    I am inserting to sql server. Error is not coming here..


    my code is not parsing.

    Yes. text file is tab delimited.

    How can i parse tab delimited text file?
    Inspiring
    February 13, 2009
    Amm85 wrote:
    > I need parse text file and insert to DB.
    > This code is not working....loop is breaking...

    Help us out here. Which database and what do you mean by "breaking"? Can you post a sample and the error message?


    > Text file looks like tab delimited.
    > i am using CSV parse code....

    Umm.. using code for a comma delimited file probably would not work on a tab delimited file.