What I am currently considering as the best option is to run tests on a simulated network we have control over (I understand there are tools for simulating network characteristics on FreeBSD routers/gateways) for some different video stream types (e.g. H.263+ with common annexes, MPEG-2, H.263, specific tools) and using that to generate a model which represents "quality" based on the network characteristics. Either that, or use probability and the known effects of loss, delay, or jitter on specific bits of data to generate a probabilistic model. The latter is more complex though.

Then, given those models, the end-user could simply run some simple ping-type tests with different packet sizes to figure out the current characteristics of their connection and then match that up with the pre-computed model associated with their codec/application.

Ketan suggested an interesting approach to this where an end-user could point their web browser to a special server that would be able to run these special ping-type tests. This would not require an end-user to even know what pinging means. Of course, this only observes the connection between the end-user and the special server, not between the two end-users who will be communicating. But it's better than nothing. It would also be possible to conduct this test several times each day over a few weeks to generate aggregate characteristics for a specific end-user and let them know what they can expect, average-case (maybe also best- and worst-case).

On Friday, September 6, 2002, at 07:14 AM, Tyler Johnson wrote:

Wesley I did a cursory review of the information you sent out regarding evaluating quality for compressed video. I suspect we are going to face an architectural decision and I would like to get you thinking about it now.

I see two possible approaches to predicting whether a particular network link will perform well for compressed video. The first is to relate network performance to perceptual quality. So we would say, for example, 5% packet loss causes sufficient tiling at target data rates/codecs that most people would regard as unusable. This would be an extremely convenient thing to do. Perhaps there would be a plug-in architecture that let's you insert your own rating scales. The down side is of course that every codec implementation is different and deals with packet loss, jitter, buffering in ways that could render those correllations meaningless.

The other approach would be to actually send sample video and analyze it on the other end. On the surface, this would seem to ensure dead accurate results. The downside is that the tool would be bigger with more components. I also wonder if we generate sample video, do we have the same problem as approach #1 in terms of different codecs behaving differently than the sample.

I would like for you to think about these issues and what the tradeoffs might be to going down either path. Perhaps there is some other approach?

Post a comment