Also added some comments and changed the callback userdata argument
to be an HttpOpRequest rather than a libcurl handle.  Less code,
less clutter.

sort things out or use policy classes (eventually) to arrange low
and high priority traffic.  Subjectively, I think this works better
in practice (as I haven't implemented a dynamic priority setter yet).

Think I have found the major factor that causes the Linksys WRT54G V5 to
fall over in testing scenarios:  DNS.  For some historical reason, we're
trying to use libcurl without any DNS caching.  My implementation echoed
that and implemented it correctly and I was seeing a DNS request per request
on the wire.  The existing implementation tries to do that and has bugs
because it is clearing caching DNS data querying only once every few
seconds.  Once I started emulating the bug, comms through the WRT became
much, much more reliable.

Data problems after connections are established should be retried as well.
Extend to appropriate libcurl codes.  Also allow our connectivity to drop
to as low as a single connection when trying to recover.

The fetch state machine received a new timeout during the WAIT_HTTP_REQ
state.  For the integration, rather than jump the state to done, we issue
a request cancel and let the notification plumbing do the rest without
any race conditions or special-case logic.

Big delta was converting the new texture debugger support code
to the new library.  Viewer manifest should probably get an eyeball
before release.

SH-3184/SH-3221  More work on cleanup with better unit test work and more aggressive shutdown of a thread.
Some additional work let me enable a memory check for the clean shutdown case and
generally do a better job on other interfaces.  Request queue waiters now awake
on shutdown and don't sleep once the queue is turned off.  Much better semantically
for how this will be used.

With this commit, the cleanup paths should be production quality.  Unit tests have been
expanded to include cases requiring thread termination and cleanup by the worker thread.
Special operation/request added to support the unit tests.  Thread interface expanded
to include a very aggressive cancel() method that does not do cleanup but prevents the
thread from accessing objects that will be destroyed.

Groundwork is used for the default class which currently represents
texture fetching.  Class options implemented from API user into
HttpLibcurl.  Policy layer is going to start doing some traffic
shaping like work to solve problems with consumer-grade gear.
Need to have dynamic aspects to policies and that starts now...

Seems to be working correctly.  Not certain this is the fastest possible way
to provide a std::streambuf interface but it's visually acceptable.

Initial version that should have enough of the plumbing to produce
a working adapter.  Memory test is showing 8 bytes held after one
of the tests so I'm going to revisit that later.  But basic
functionality is there going by the unit tests.

Only thing interesting in this changeset is the discovery that a sleep
in the fake HTTP server ties up tests.  Need to thread that or fail on
client disconnect or something to speed that up and make it usable for
bigger test scenarios.  But good enough for now...

When a Content-Range header is received, make available the full triplet of <offset, length, fulllength>.

Pretty straightforward.  Still don't like how I'm managing
the options block.  Struct?  Accessors?  Can't decide.  But
the options now speed up the unit test runs even as I add
tests.

With much trial-and-error, cleaned up the banner on the texture console and made everything
mostly fit.  Added global cache read, cache write and resource wait count events to the
console display to show if cache is working.  On clean exit, emit a log line to report
stats to log file (intended for automated tests, maybe):
LLTextureFetch::endThread: CacheReads:  2618, CacheWrites:  117, ResWaits:  0, TotalHTTPReq:  117

Beefed up the metrics gathering in http_texture_load to get memory sizes and
cpu consumption on windows (still need to implement that on Mac & linux).
Ran runs with various idle loops with sleeps from 20 ms down to pure spinning,
varied Block allocation size from 1504 to 2^20 bytes.  2ms/2ms/65540 appears
to be a good spot under the test conditions (Win7, danu grid, client in Boston).

This is a command-line utility to pull content down from a service through
the llcorehttp library to produce timings and resource footprints.

LLMutex recursive lock, global & per-request tracing, simple GET request, LLProxy support, HttpOptions starting to work, HTTP resource waiting fixed.
Non-LLThread-based threads need to do some registration or LLMutex locks taken out in these
threads will not work as expected (SH-3154).  We'll get a better solution later, this fixes
some things for now.  Tracing of operations now supported.  Global and per-request (via
HttpOptions) tracing levels of [0..3].  The 2 and 3 levels use libcurl's VERBOSE mode
combined with CURLOPT_DEBUGFUNCTION to stream high levels of detail into the log.  *Very*
laggy but useful.  Simple GET request supported (no Range: header).  Really just a
degenrate case of a ranged get but supplied an API anyway.  Global option to use the
LLProxy interface to setup CURL handles for either socks5 or http proxy usage.  This
isn't really the most encapsulated way to do this but a better solution will have to
come later.  The wantHeaders and tracing options are now supported in HttpOptions giving
per-request controls.  Big refactoring of the HTTP resource waiter in lltexturefetch.
What I was doing before wasn't correct.  Instead, I'm implementing the resource wait
after the Semaphore model (though not using system semaphores).  So instead of having
a sequence like:  SEND_HTTP_REQ -> WAIT_HTTP_RESOURCE -> SEND_HTTP_REQ, we now
do WAIT_HTTP_RESOURCE -> WAIT_HTTP_RESOURCE2 (actual wait) -> SEND_HTTP_REQ.  Works
well but the prioritized filling of the corehttp library needs some performance
work later.

The NORMAL range doesn't do any sleeping at all and so we'll
spin the core harder than we already are.  Bring all idlers
into the same range.

For now, workaround...

Implemented/modified PUT & POST to not used chunked encoding for the request.
Made the unit test much happier and probably a better thing for the pipeline.
Have a cheesy static & dynamic proxy capability using both local options and
a way to wire into LLProxy in llmessages.  Not a clean thing but it will get
the proxy path working with both socks5 & http proxies.  Refactoring to get
rid of unneeded library handler and unified an HttpStatus return for all
requests.  Big batch of code removed as a result of that and more is possible
as well as some syscall avoidance with a bit more work.  Boosted the unit
tests for simple PUT & POST test which revealed the test harness does *not*
like chunked encoding so we'll avoid it for now (and don't really need it
in any of our schemes).

Beware of bad documentation.  operator--(int) does not return what
the header claimed it did.

This brings in a copy of llmessage's llsdmessage testing server.  We run
a mocked HTTP service to handle requests and the integration tests run
against it by picking up the LL_TEST_PORT environment variable when running.
Add some checks and output to produce useful info when run in the wrong
environment and when bad status is received.  Later will add a dead port
as well so we can test that rather than use 'localhost:2'.

Generally, opaque data operations are expected to be over 'void *' and have
now converted interfaces to do that.  Update() method honors millisecond limit to dwell
time.  Might want to homologate the millis/uSecs mix later....