video and audio latency from 6 seconds to 4 seconds

Hello,

I have developed a video transmission and viewing application . I am facing a problem of one way video latency of about 6 seconds.

Adobe says that the latency can be reduced to about 3 seconds one way. I have set up the buffertime for transmitter about 3 seconds.

Please can  anyone suggest me how to reduce the latency from 6 seconds to about 3 seconds ??

I have no issues regarding the bandwidth , but can anyone suggest the proper optimization technique for CPU and memory usage .

What should be the ideal configuration for minimum latency. 

Currently my audio and video  transmission one way is 6 sec and hence 2 way for both is 12 -14 sec.

Kindly guide me how to resolve the problem.

Thanks,

Chinmay Chitale