Script to integrate local LLM for proofreading

[alexanderp3125138]

Hey folks,

I'm thinking of writing a script to add local LLM api support to FrameMaker. I basically want to get AI assisted proofreading/text correction by doing the following (using a local lightweight Mistral-7B model):

1. Select text, then send it via button press to an LLM client  running on my computer (LM Studio or ollama) for proofreading
2. Receive & process the proofreading results
3. Apply the suggestions to the text in FrameMaker

Before I dive into what might just be an impossible task due to ExtendScript limitations (I'd have to do quite a lot of digging, I'm not a real coder), do you think this is doable? I'd be very grateful for any insights, or tips where to start. 🙂🙏

[frameexpert]

How can your LLM receive inputs? You could send text via a commandline from ExtendScript. ExtendScript does allow some integration with other programs via C++.

[alexanderp3125138]

Input is received via local api (LM Studio = http://localhost:1234/, ollama = http://localhost:11434/). I'm wondering if it's possible to create some sort of automated "roundtrip", with text pushed to the api, and completion received automatically.

[frameexpert]

Do you have any code examples of sending and receiving data with LM Studio?

[alexanderp3125138]

They have several examples listed in LM Studio. It's all based on using the OpenAI API

hello world (curl):

curl http://localhost:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
{ "role": "system", "content": "Always answer in rhymes." },
{ "role": "user", "content": "Introduce yourself." }
],
"temperature": 0.7,
"max_tokens": -1,
"stream": false
}'

chat (Python):

# Example: reuse your existing OpenAI setup
from openai import OpenAI

# Point to the local server
client = OpenAI(base_url="http://localhost:1234/v1", api_key="not-needed")

completion = client.chat.completions.create(
model="local-model", # this field is currently unused
messages=[
{"role": "system", "content": "Always answer in rhymes."},
{"role": "user", "content": "Introduce yourself."}
],
temperature=0.7,
)

print(completion.choices[0].message)

ai assistant (Python):

# Chat with an intelligent assistant in your terminal
from openai import OpenAI

# Point to the local server
client = OpenAI(base_url="http://localhost:1234/v1", api_key="not-needed")

history = [
{"role": "system", "content": "You are an intelligent assistant. You always provide well-reasoned answers that are both correct and helpful."},
{"role": "user", "content": "Hello, introduce yourself to someone opening this program for the first time. Be concise."},
]

while True:
completion = client.chat.completions.create(
model="local-model", # this field is currently unused
messages=history,
temperature=0.7,
stream=True,
)

new_message = {"role": "assistant", "content": ""}

for chunk in completion:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
new_message["content"] += chunk.choices[0].delta.content

history.append(new_message)

# Uncomment to see chat history
# import json
# gray_color = "\033[90m"
# reset_color = "\033[0m"
# print(f"{gray_color}\n{'-'*20} History dump {'-'*20}\n")
# print(json.dumps(history, indent=2))
# print(f"\n{'-'*55}\n{reset_color}")

print()
history.append({"role": "user", "content": input("> ")})

[frameexpert]

ExtendScript has a Socket object. I haven't used it, but here is an excerpt from the ExtendScript documentation.

[alexanderp3125138]

Thanks, I'll take a closer look. If I can cook something up, I'll share my findings here. 🙂

[Russ Ward]

The ExtendScript implementation is very low-level and intolerant to errors, but it works. You basically have to build the HTTP request character by character and a single formatting error can break it. I have an ExtendScript sample here that demonstrates how to open a socket and do a GET request (10.02 NETWORK_-_Retrieve_text_over_network.jsx):

http://weststreetconsulting.com/WSC_ExtendScriptSamples.htm

In your case, you would need to do a POST request. I have never done that, but I'm thinking it would be about the same... you would just need to add the POST body and probably some headers. That's just a guess, though. In any case, I hope this helps.

Russ

[alexanderp3125138]

I'll just throw this in here as WIP. Here's what's happening:

1. I can't get the script to actually add my text selection to the JSON string, there's simply nothing happening.

2. If I take the "var highlighted text" snippet out, the script does send the system prompt to LLM studio successfully, which then generates an answer, and tries to send it back.

3. In that case, I then receive an error message in FrameMaker ("Corrected Text: undefined"), which means something is received, but not in the correct format.

Anway, here's my code so far (please bear with me, I'm no coder and am trying to piece this together somehow 😇

[Edit: code deprecated & removed, update further below]

[Russ Ward]

Hi, I definitely commend you for trying to do this... much of the value of FrameMaker is found in the ability to customize it and very few users actually leverage this value. So I encourage you to keep going. Having said that, you are attempting to do a variety of complicated things all at once and I think you need to back up, study a little more, and go piece by piece. For example, right in the beginning you have "app.activeDocument.selection", but neither the "activeDocument" nor "selection" properties actually exist. I could be wrong, but I suspect that nobody is willing to spend lots of time giving free ExtendScript training, so my recommendation is to back up a bit, learn the individual pieces, then come back if you have specific questions. I hope you don't take this the wrong way... I just think it is the best approach for anyone of they want to succeed at something so ambitious and also get productive help from the people who volunteer their time here.

Russ

[frameexpert]

I usually take a bottom up approach when writing a script; I divide the overall task into small tasks, get each one working, and then combine them into my main script. In this case, I might have:

• Get the selected text as a string
• Make a successful connection to the server
• Send a generic string (not the selection) to the server and get a response

I usually start with an active document and my cursor in a paragraph (or a table, or whatever I am working with). After everything is working at that level, then I expand it out to the document or book level.

Another advantage to this approach is that you can make separate posts here on the forum and people are more likely to help you with smaller tasks. It also helps other users that might just be interested on one of your smaller tasks.

[frameexpert · Accepted solution]

I gave you a headstart with this post:

https://community.adobe.com/t5/framemaker-discussions/get-framemaker-text-as-a-string/td-p/14491503

You will see that I have created a function for getting text from any FrameMaker text object (TextRange, Pgf, Flow, etc.). Functions like this are useful because you can plug them into any of your scripts as needed.

[alexanderp3125138]

Slowly getting there... my script does send highlighted text over the api now, and processes the corrected text. I have two issues left, any help would be greatly appreciated:

1. the returned text always replaces the whole paragraph instead of the highlighted text, and 

2. Sometimes I get a weird error from this snippet:

alert("Failed to parse JSON response: " + e.toString()); 

The error message is "reference error: ) has no value". Is that maybe related to the charset, i.e. is my llm returning unsupported characters?

Here's my updated code, I'll truncate/delete my post from above, as it's not relevant anymore:

var doc, postData, postRequest, conn, response, bodyStart, body, jsonResponse, correctedText;

// Make variables for the active document and the current text selection.
doc = app.ActiveDoc;

if (doc.ObjectValid() == true) {
    var textRange = doc.TextSelection; // TextRange object
    var highlightedText = getText(textRange, doc); // Use the getText function to get the highlighted text

    if (highlightedText !== "") {
        // Prepare your JSON payload using the highlighted text
        postData = JSON.stringify({
            "messages": [
                {
                    "role": "user",
                    "content": "You are a precise and accurate proof reader. You only ever give the corrected text, but no further explanation! Correct the following text: " + highlightedText
                }
            ]
        });

        // Create the POST request
        postRequest = "POST /v1/chat/completions HTTP/1.1\r\n" +
            "Host: localhost:1234\r\n" +
            "Content-Type: application/json; charset=utf-8\r\n" + // Specify UTF-8 charset
            "Content-Length: " + postData.length + "\r\n" +
            "\r\n" +
            postData;

        conn = new Socket;
        // Attempt to open our socket connection
        if (conn.open("localhost:1234", "binary")) {
            conn.write(postRequest);

            // Response reading
            response = conn.read(999999); // Adjust as necessary
            conn.close();

            // Parsing
            bodyStart = response.indexOf("\r\n\r\n") + 4;
            body = response.substring(bodyStart);

            // Assuming JSON response
            try {
                jsonResponse = JSON.parse(decodeURIComponent(escape(body))); // Decode the response body
                correctedText = jsonResponse.choices[0].message.content; // Adjust based on actual JSON field
                alert("Corrected Text:\n\n" + correctedText);

                // Start replacing the text as per the example provided
                // First, select the text to be replaced
                textRange.beg.obj = textRange.end.obj = textRange.beg.obj; // Assuming the text range is within a single paragraph/object
                textRange.beg.offset = 0;
                textRange.end.offset = Constants.FV_OBJ_END_OFFSET;
                doc.TextSelection = textRange;
                textRange = doc.TextSelection;
                textRange.end.offset--;
                doc.TextSelection = textRange;

                // Clear the selected text
                doc.Clear(0);

                // Insert the corrected text
                doc.AddText(textRange.beg, correctedText);

            } catch (e) {
                alert("Failed to parse JSON response: " + e.toString());
            }
        } else {
            alert("Failed to open a connection to the API.");
        }
    } else {
        alert("No text is highlighted or selected. Please highlight the text you want to correct.");
    }
} else {
    alert("No active document found. Cannot continue.");
}

alert("Script complete!");

[frameexpert · Accepted solution]

// Replace the selected text.
// You already have the selection in a varable.
doc.DeleteText (textRange);
doc.AddText (textRange.beg, correctedText);

Untested, but it should work.

[alexanderp3125138]

Thanks, will try this as soon as I can get the script to run again. This is driving me insane, FrameMaker suddenly complains about the JSON.stringify section with a "JSON is undefined" error. It was all running fine on Friday, but when I ran the same script again yesterday, it bugged out. Ah well, back to the drawing board, I guess. 😉 

[alexanderp3125138]

Quick update: it's working again. I put the script back together piece by piece, which didn't work. Then I transferred it back to the PC I had been working on last Friday, and there, it just went through without any error message. Weird thing that FrameMaker can execute that JSON part just fine on one machine, but not on another, but that's probably some weird configuration thing. ¯\_(ツ)_/¯
Anyway, thank you very much for the help! 👍

[frameexpert]

A good next step would be to break each part of the script down into individual functions. It makes things more modular and gives you some reusable building blocks. It should also make it easier to troubleshoot because you can test individual parts of the script to see where its failing.

[alexanderp3125138]

Yes, that's one thing I want to do. I just kinda frankensteined this together, more or less learning on the fly how to use ExtendScript.

Going forward from here, I'd like to try and find a way to loop through a whole document, and send the contents paragraph by paragraph to the proofreader, so they get transferred back into their source paragraph format. Something like this:

Title -> to LLM -> back to document as title format

Heading 1 -> to LLM -> back to document as Heading 1 format

Standard text body -> to LLM -> back to document as standard text

Heading 2 -> etc. pp.

If (alright, that's a big "if") the output quality from the llm is good and reliable enough, I could just let my computer do the proof reading, and do something productive in the meantime. 😉

[alexanderp3125138]

...aaaand a (hopefully small) follow-up issue: I'm getting a "client disconnected" message in LM Studio after about 10 seconds, which cancels the whole process. It's basically a timeout issue. A quick&dirty workaround I tested was "$.sleep(20000);" after the "conn.write(postRequest);" , which gave it enough time to finish a longer text. But of course that (a) freezes FrameMaker for the duration of the sleep command, and (b) isn't flexible and doesn't account for even longer texts.
Is there a way to polling for the response being sent? I've searched this forum, but didn't find a working fix. 😞

[frameexpert]

I am not sure the answer to your question, but alternative is to see if the API will take a complete XML file and send one back. Then you could write an XML file, send if off, receive the translation, and then write the XML back to FrameMaker. This is a whole different task, but may solve the connection issue of sending them one at a time.

[alexanderp3125138]

So, I've managed to implement a working fake polling mechanism by simply putting the script to sleep in a 100ms loop, and checking for incoming data in between. It's more or less working, but I'm facing an issue:

To proof-read the whole document without exporting and reimporting everything, the only option I imagine could work is to select and then send each paragraph to the api. My problem is that I can't for the life of me figure out how to select text with ExtendScript. I've been looking up script examples, searched the web, but nothing works. I'm guessing it must be some variation of this:

var firstParagraph = doc.FirstPgf;

doc.TextSelection = new TextRange(firstParagraph, firstParagraph.EndOfContent);

Not sure though, and I don't want to select the first paragraph, but the next. Is there a kind soul willing and able to enlighten me? 🙂

[frameexpert]

Here is a function I use to get text from a text object such as a Pgf, Cell, etc. For example to get the text from paragraph containing the cursor, use something like this:

var doc, pgf, text;

doc = app.ActiveDoc;

pgf = doc.TextSelection.beg.obj;

text = getText (pgf, doc);

alert (text);

function getText (textObj, doc) {

    // Gets the text from the text object.

    var textItems, text, count, i;
    
    // Get a list of the strings in the text object or text range.
    if (textObj.constructor.name !== "TextRange") {
        textItems = textObj.GetText (Constants.FTI_String);
    } 
    else {
         textItems = doc.GetTextForRange (textObj, Constants.FTI_String);
    }
    
    // Concatenate the strings.
    text = "";
    count = textItems.length;
    for (i = 0; i < count; i += 1) {
        text += (textItems[i].sdata);
    }
    
    return text;
}

[alexanderp3125138]

Thanks for your reply, I really appreciate it. 🙂

I'm looking for a way to actually select a paragraph with Extendscript. The idea is that I want the user to be able to either select text with the mouse, or, if nothing is selected, simply select the whole next paragraph. This kind of advanced either/or functionality will be reserved for later though. I'll stitch it together once both parts are working independently. 

Right now, option 1 is working fine, the text is sent over to the llm via api, connection is staying alive as long as needed, corrected text is received and pasted back into the document, overwriting the previous (selected) text. 

Now for option 2, if the text gets copied and sent over automatically without selecting it first, it gets corrected, too, but then inserted into the paragraph at insertion point, so I end up with the old and new text.

So all I need is a way to select the next paragraph relative to the cursor position. I think I could integrate this easily with the getText function that's already working. I could also paste my whole script text here, but that would probably just unnecessarily blow up my post, and I don't want to bother you guys even more with walls of text. 😉

[frameexpert · Accepted solution]

These two short videos will explain text locations and text ranges:

https://youtu.be/6ywYWU4VC7g?si=pA7llQ5vS8nIlqwt

https://youtu.be/fH5giAcDp6Q?si=lerkUNDTWdSoLKjO

If you are in a hurry, the second one will show you how to select a paragraph.

[alexanderp3125138]

That did the trick, thank you very much! I also adjusted the script to perform a check if text has already been selected. So it will allow you to either select+correct or just correct the next paragraph.

Now I only need to figure out how to reduce the initial sleep/wait period when sending data via the api. It usually gets corrected within a second or two, and then FrameMaker "hangs" until it processes the received data, i.e. pastes it into the document.

Here is my revised WIP script, if anyone is interested. Should be compatible with any OpenAI api, and the LLM behind it can be configured to do pretty much anything, from simple proof reading to improving overall language quality, or doing some more crazy stuff, like code correction or something:

var doc = app.ActiveDoc;

main();

function main() {
    if (doc && doc.ObjectValid() === 1) {
        processDoc(doc);
    }
}

function processDoc(doc) {
    var textRange, highlightedText;

    // Check if text is already selected
    if (doc.TextSelection.beg.offset !== doc.TextSelection.end.offset) {
        textRange = doc.TextSelection;
        highlightedText = getText(textRange, doc);

    } else {
        // If no text is selected, select the next paragraph
        var textLoc = doc.TextSelection.beg;
        var pgf = textLoc.obj;
        var nextPgf = pgf.NextPgfInFlow;

        if (nextPgf !== null) {
            textRange = new TextRange(
                new TextLoc(nextPgf, 0),
                new TextLoc(nextPgf, Constants.FV_OBJ_END_OFFSET - 1)
            );

            doc.TextSelection = textRange;
            highlightedText = getText(textRange, doc);
        } else {
            alert("No next paragraph found.");
            return;
        }
    }

    // Popup alert to inform user that text is being corrected
    alert("Text is now being corrected.");

    if (highlightedText !== "") {
        var postData = JSON.stringify({
            "messages": [
                {"role": "user", "content": "Correct the following text: " + highlightedText}
            ]
        });

        var postRequest = "POST /v1/chat/completions HTTP/1.1\r\n" +
                          "Host: localhost:1234\r\n" +
                          "Content-Type: application/json; charset=utf-8\r\n" +
                          "Content-Length: " + postData.length + "\r\n" +
                          "\r\n" + postData;

        var conn = new Socket;
        if (conn.open("localhost:1234", "binary")) {
            conn.write(postRequest);
            var response = "";

            while (response.length === 0) {
                response = conn.read(999999);
                if (response.length === 0) {
                    $.sleep(100);
                }
            }

            conn.close();
            var body = response.substring(response.indexOf("\r\n\r\n") + 4);
            var jsonResponse = JSON.parse(decodeURIComponent(escape(body)));
            var correctedText = jsonResponse.choices[0].message.content.replace(/\n/g, "\r\n");

            // Insert corrected text
            doc.DeleteText(textRange);
            doc.AddText(textRange.beg, correctedText);
        } else {
            alert("Could not establish connection to the API.");
        }
    } else {
        alert("No next paragraph. End of document?");
    }
}

function getText(textObj, doc) {
    var textItems = doc.GetTextForRange(textObj, Constants.FTI_String);
    var highlightedText = "";

    for (var i = 0; i < textItems.length; i++) {
        highlightedText += textItems[i].sdata;
    }

    return highlightedText;
}

// Script finished notification
alert("Script executed");