Re: POST procedure (squid 2.7 stable9)

Please describe your goal in terms of HTTP methods and results, not
abstract "twitter like". It will be very hard to help you if not
understanding what you want to accomplish at the HTTP level.

Discussing this based on twitter is pretty meaningless. It's not twitter
(at least not today), and it also does not say anything about what the
sequence of HTTP operations and their expected results may look like.

You mention that the requests may be send to the web server later. So
what you want to implement is a POST queue?

When will the POST queue be run?

What response should be sent to the requesting clients? Normally they
expect a response from the requested web server.

What to do if the web server transaction then fails?

Regards
Henrik

mån 2012-03-19 klockan 08:54 +0800 skrev Schulz Xu:
> My goal is simply to save the content of HTTP post.
> For example, when a user twitter a tweet. The client will send a POST
> request to the Twitter.
> If the Twitter uses the squid as proxy servers, squid will usually
> forward this HTTP request to the web server. Further, the web server
> will save the content which indicates a successful upload procedure.
> But, what I want to is to save the content in HTTP Post(i.e. the
> tweet) on squid.
> When receive the POST, first analyze the request then call StoreIo...
> (I am tryin' to figure it out)to store the content in the squid file
> system.
>
>
> Why I am doing this is because I figure that for many Twitter-like
> websites, they are often confronted with high income of POST requests
> at certain special moment(Super Bowl, etc.). If we manage to use the
> proxy servers to preserve the POST than to forward to the web server,
> we can largely reduce the pressure on web servers since we distribute
> the requests on many proxy servers. The content that saved on squid
> can be packed and then sent to the web server later.
> I hope I made point clear.
>
> 在 2012年3月18日 下午9:59，Henrik Nordström
> <henrik_at_henriknordstrom.net>写道：
> sön 2012-03-18 klockan 21:19 +0800 skrev Schulz Xu:
> > Thanks for the tips. I trace back from the
> >
> ProcessRequest->clientAccessCheck->ClientFinishRewrite->ClientStoreURLRewrite->clientRedirect->clientAccCheckDone
> > On the other hand, I follow from tryParse to
> > clientCheckFollowXForwardFor. But other problem occur.
> > 1. in clientCheckFollowXForwardFor, what is
> FOLLOW_X_FORWARDED_FOR???
>
>
> It's a define set by configure, enabled by the
> --enable-follow-x-forwarded-for configure options. If enabled
> then
> FOLLOW_X_FORWARDED_FOR is true and the related code gets
> compiled. If
> not enabled then it's false/undefined and the related code do
> not get
> compiled.
>
> > 2. no matter how, the procedure will finally lead to the
> > clientAccessCheck-->aclNBCheck-->aclCheck--> then I am
> kinda lost in
> > the codes couldn't find the way from function aclCheck to
> function
> > clientAccessCheckDone (which I think is the process.)
>
>
> aclNBCheck calls the given callback on completion, i.e.
> clientAccessCheckDone in case of clientAccessCheck().
>
> aclNBCheck(http->acl_checklist, clientAccessCheckDone,
> http);
>
> processes the given checklist, then calls
> clientAccessCheckDone with the
> result and http as argument.
>
> > 3. If I am about to INTERCEPT the POST request and manage to
> store the
> > body-content in the file system(I'll start with ufs but I
> hope that I
> > can utilize the COSS),
>
>
> ufs and coss uses the same interface, only implementation
> differ.
>
> > what procedure you think I should follow. At the moment,
> I'd like
> > examine the code in processReq. Cuz usually when tagged as
> MISS, the
> > squid will be asked to forward the POST. However,can you
> tell me a
> > simple procedure to store the content in the squid?
>
>
> I am not entirely sure what you want to do with stored POST
> data.
>
> What is it from an HTTP layer point of view that you want to
> accomplish?
> Please describe your goal in terms of HTTP requests and their
> responses.
>
> Regards
> Henrik
>
>
>
> >
> >
> > Thank you( and your time)
> >
> > Schulz
> >
> >
> > 在 2012年3月16日 下午9:33，Henrik Nordström
> > <henrik_at_henriknordstrom.net>写道：
> > tor 2012-03-15 klockan 18:40 -0700 skrev Schulz:
> >
> >
> > > Simply speaking, when dealing with the POST
> request, squid
> > mainly use
> > > the readRequest-->tryParseHttp-->parseHttpReq(in
> which using
> > the
> > > urlParse to find the method )
> >
> >
> > That's the general request parsing path, parsing the
> request
> > into
> >
> > Request line
> > Request headers
> > Body pipe (not parsed, only transferred)
> >
> > this is the same for all requests with POST no
> different from
> > a
> > GET/HEAD/OPTIONS/PUT/DELETE/whatever. Even CONNECT
> gets parsed
> > the same
> > way.
> >
> >
> > > When parseHttpReq retrieve all the useful info,
> then the
> > tryParse
> > > would react according to the http. But tryParse
> only return
> > the bytes
> > > it used to the readRequest. Where is the following
> > procedure?
> >
> >
> > When the request is parsed it's added to the list of
> pending
> > requests on
> > the connection and handed off to the chain of access
> controls.
> >
> > In Squid-2 the access control chain starts with
> > clientCheckFollowXForwardedFor() and ends in
> > clientProcessRequest(),
> > where it checks it it's a cache etc.
> >
> > then if the request needs to be forwarded (which is
> always the
> > case for
> > POST) it calls fwdStart(), which does a bit of a
> dance to
> > figure the
> > right path for forwarding the request, opens a
> connection and
> > hands off
> > to the respective protocol handler, i.e. http.
> >
> > Up to this point the body of the POST request is not
> touched
> > at all.
> > Only the protocol handler accesses the body of the
> POST
> > request.
> >
> > Regards
> > Henrik
> >
> >
> >
> >
>
>
>
>
>
Received on Mon Mar 19 2012 - 03:27:48 MDT