Detect Audio with JavaScript (not Actions)

[sharonhuston23]

I'm working on an external script (not an action) that will parse a timed text file to display closed captions in Captivate. It's about halfway finished -- I can parse the file, and load the captions into a <div> I created.  (Yippie!)  The problem is that I want the captions to start automatically when the audio on the timeline plays. Right now the captions start when a new slide loads.  Sometimes, however, our audio doesn't start immediately, so everything gets out of sync. 

The Common JavaScript Interface doesn't seem to have anything I can use, so I've resorted to playing with the CMP file, and I'm completely lost.  Can anyone help me get started?  Does anyone know what part of the code controls audio?

Thanks,

Sharon

[sharonhuston23]

I'm making very slow progress.

• cp.model.audios is an object that holds the name of the audio files.
• cp.model.data holds several audio objects, all named with a pattern StAd#.

Neither of these objects seems to associate a slide with a particular audio file.  That association would be helpful.  At some point I'll want to say something like "If the audio on Slide 5 is playing, then show the closed captions."  It would help to know what Captivate named the audio on Slide 5.

I also found an AudioObject, and an AudioManager, but I'm not sure what they do.  Common sense says the AudioObject has information about a specific piece of audio, while the AudioManager plays/pauses/stops the audio, but I'm not sure yet. 

I'm also trying to manipulate the audio objects in the console using cp.stopAudio and cp.playAudio, but I'm not having any luck.  Any help is appreciated.

[eLearning_Pundits]

Hey Sharon, I checked your question earlier but didn't really gave a thought. Sorry about that.

Now, I am not into JS developer but I just read your original question a bit carefully and tried to get it work using Advance Actions. Answer to your question

The problem is that I want the captions to start automatically when the audio on the timeline plays. Right now the captions start when a new slide loads.  Sometimes, however, our audio doesn't start immediately, so everything gets out of sync. 

could be here in screenshot below. it worked for me, let me know if I get it correctly. Or may be I didn't get your question and you don't want to use advance actions.

I have applied the action below on enter while keeping the default CC disabled:

Check the preview for the same project.

https://elearningpundits.com/cp_uploads/demo_projects/

[RodWard]

To be quite honest, I think using specialised Javascripting for this is way overkill.

If you don't want to use the Closed Captioning system that is built into Captivate (which works perfectly well), and you want to add your own captions instead, then just using Object Audio on those captions would be a lot simpler that trying to code in JS.  That way the audio would be loading and playing at exactly the same time as the captions entered the timeline.

[eLearning_Pundits]

Totally agree with RodWard

[sharonhuston23]

Thank you both for the help.  I can see your point, but I find manually setting timings to be a needless time suck.

Setting the timing one slide note at a time takes about 30 minutes per lesson, depending on the size of the lesson.  Since we work from scripts, I can generate .SRT files in about five minutes using the Python library aeneas. (https://www.readbeyond.it/aeneas/ )  That's a 27 minute saving.  Multiply that by  . . . oh . . . 30 lessons (about what my department produces in a month) and we can save 13 hours a month.  That's more than worth the time, especially when considered over the course of a year.  Plus the method I'm working on will require very little re-training. 

(Sigh) There is a reason Lectora and Storyline both support .SRT files.

[sabre123]

Maybe you should log a feature request.

[RodWard]

I think there is something else that should be factored into your equation here.

YOU are obviously someone with some programming skills. You can use a Python library and scripting to save time and generate your custom Closed Captioning solution.  And that combination of technical expertise has allowed you to achieve a whole 13 hours time saving over a month.  YOU believe the method you're working with would be achievable by anyone with a little training.

You MIGHT be right about that assumption, but in my experience, you WON'T be.  (I've worked at a few software companies and I'm very familiar with the way programmers think.)

Somewhere in the future, after you have moved on somewhere else, another e-learning developer will inherit your custom-crafted e-learning solution and won't know what to do with it.  They won't have your skills.  The documentation about how to maintain the solution will have been mislaid or lost in the company's IT system (assuming you're one of those rare technical people that actually DOCUMENTS their solutions for others).  And then your wonderful solution is going to cost the company a lot more than 13 hours a month.

In fact, a very common outcome is that the entire solution gets scrapped.

While Captivate's Closed Captioning solution might not be the speediest solution out there, it IS at least eminently maintainable by any Captivate developer.  They DO NOT need to have Python scripting experience (or any programming expertise whatsoever) to maintain it. That's a big plus.

[sharonhuston23]

What you're saying is very true -- this could be a solution that's eventually abandoned.

My hope is that eventually Captivate will catch up with reality and allow users to import .SRT files, so *when* that happens (insert optimism) what I'm doing will be a stopgap measure.  I'm trying to write my code to live within the existing Captivate ecosystem.  The captions, for example, will still be in slide notes (my team is trained to write that way now) but we won't check the CC box.  The .SRT captions will (hopefully) appear in the existing Adobe CC box, but if they don't I can write to a custom <div>.  If something does explode we (or they, if I'm not around) can move backwards without too much pain.

You're also seriously overestimating the complexity of what I'm doing based on the word "Python."  The library is really simple to use, all I had to do was write a tiny batch file.  This is, in fact, the first time I've done *anything* with Python, so I'm pretty confident that programmers we have on staff could help if there is an emergency need.

Thank you for your thoughts -- it's nice to see someone try to save me from myself.

[sharonhuston23]

It's also worth noting that there are VERY FEW things we can do to speed up or automate the Captivate development process. I celebrate any opportunity that will automate a routine processes so I can instead concentrate on decisions machines can't make.

[RodWard]

I'm somewhat more interested in the creative and instructional side of the e-learning process. That part doesn't usually lend itself to automation (so far as I am aware) but it seems to be the part that has the biggest impact on whether the e-learning is engaging and interesting for the learners.

Perhaps automation is over-rated.

[leonardo_portillo_contractor]

I am very very sorry @RodWard to meddle in this, don't get me wrong because this is not personal but... wow! I usually wander around these posts looking for answers before asking questions. I am a developer, you have already read from me. As I have stated before, I'm using Captivate only because I'm being paid to do so. I would have chosen another tool. Nevertheless, I'm very glad about what I have achieved by using JS within Captivate, despite how poorly documented Captivate is. I've been able not only to simulate a procedure but a system itself with data, relationships, user inputs, dates, calculations, etc. There's a world of things we can only do with JS within Captivate.

You obviously have very little understanding of how professional developers work because you assume that almost nobody documents their code or make it maintainable and portable by others. That's actually pretty fundamental to serious coding. There are even specialized tools to do so, as well as reengineering and refactoring techniques. I don't know what kind of developers you have been working with but, what you say is so far from the truth. By giving what you say for granted, writing code is worthless because it will always end up being scrapped. You're so wrong and have a very limited understanding of it. Writing code doesn't always have to be about making things easier but making this BETTER and doing MORE. I think you need to understand that and have some more respect for developers. You tend to set the tool over the users, implying that users do better limiting themselves to the tool. Be thankful that there are inquisitive developers who always strive to push the boundaries, they do make a difference in the long run, the tool doesn't. I personally love it when somebody who is not a developer says: don't do that, it won't work. That's usually a clue that I am on the right track because I'm being able to see something that others can't see. Don't disparage or divert others' questions. Let others just ask and get answers. And don't make things personal by writing uppercase things such as YOU, MIGHT, WON'T, which is very impolite. 

My point is... in a discussion like this, you have diverted what was a technical question into an almost pointless discussion about what you think of developers, what MIGHT happen in a company or business you don't know, how much expertise a person you don't know has, or if automation is overrated. I personally work with a team in which others take care of the creative and instructional side of e-learning, and I take care of the technical side. It doesn't make my job any less valuable or my questions any less worthy to be answered. If anybody asks for a "technical detail" that you don't think is useful.... will you come up every time with a lecture about why not to do what they want to do? sharonh34578565 just asked a simple question in which I'm interested too and nobody has tried to answer in this long discussion: Does anyone know what part of the code controls audio?

Please, if by any chance anybody has an answer by these days, I would truly appreciate it... Does anyone know what CP Javascript functions control audio and/or is there a reference?

I'm very sorry to sound harsh, don't get me wrong. Thank you very much in advance!

[RodWard]

@ Leonardo. Not sure what I have done to earn so much of your vitriol, but I DO see your attack as a personal one, and I don't intend to waste my time on replying to it.

[TLCMediaDesign]

If you wanted to do what the OP wanted, you just need to wrap getElementsByTagName("AUDIO")

in a very short timeout so that it can be rendered in the DOM, attach standard timeupdate event listeners to it and do what needs to be done.

You can parse through tons of info just by logging cp, cp.movie.

By the way, @RodWard 's son is an outstanding developer/programmer.

[leonardo_portillo_contractor]

Thank you very much @TLCMediaDesign for shedding some light on the real issue. By now, I'm trying to make use of cp.playAudio() on conditional JS actions but can't make it work. Chrome's JS console throws a hint that cp.playAudio() takes two arguments: f(c, b ). I've been trying to figure them out with no success. Please, do you know what they are? Do you know how to use the window.cp object to control audio? Thank you in advance!

[TLCMediaDesign]

Are you talking about slide audio or object audio?

[Jeremy Shimmerman]

When you import an audio file into Captivate, it renames it as a random number and stores it in the /ar folder.

If you want to control audio using js, you need to manually place your .mp3 files in the /ar folder.  The script to controlling the audio is:

cp.playAudio('filenamewithoutsuffix')

cp.stopAudio('filenamewithoutsuffix')

Hope that helps.  

[Jacob Hokland]

And if you replace your audio files in the SCORM package afterwards you will also be able to deliver better sound quality as the files will not be further compressed.

/Jacob