Parent Directory
|
Revision Log
| Links to HEAD: | (view) (download) (annotate) |
| Sticky Revision: |
Add Multipath TCP (MPTCP) support (Proxy) Multipath TCP (MPTCP), standardized in RFC8684 [1], is a TCP extension that enables a TCP connection to use different paths. Multipath TCP has been used for several use cases. On smartphones, MPTCP enables seamless handovers between cellular and Wi-Fi networks while preserving established connections. This use-case is what pushed Apple to use MPTCP since 2013 in multiple applications [2]. On dual-stack hosts, Multipath TCP enables the TCP connection to automatically use the best performing path, either IPv4 or IPv6. If one path fails, MPTCP automatically uses the other path. To benefit from MPTCP, both the client and the server have to support it. Multipath TCP is a backward-compatible TCP extension that is enabled by default on recent Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath TCP is included in the Linux kernel since version 5.6 [3]. To use it on Linux, an application must explicitly enable it when creating the socket. No need to change anything else in the application. Adding the possibility to create MPTCP sockets would thus be a really fine addition to httpd, by allowing clients to make use of their different interfaces. This patch introduces the possibilty to connect to backend servers using MPTCP. Note however that these changes are only available on Linux, as IPPROTO_MPTCP is Linux specific for the time being. For proxies, we can connect using MPTCP by passing the \"multipathtcp\" parameter: ProxyPass \"/example\" \"http://backend.example.com\" multipathtcp=On We then store this information in the worker and create sockets appropriately according to this value. Link: https://www.rfc-editor.org/rfc/rfc8684.html [1] Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2] Link: https://www.mptcp.dev [3] Add Multipath TCP (MPTCP) support (Core) Multipath TCP (MPTCP), standardized in RFC8684 [1], is a TCP extension that enables a TCP connection to use different paths. Multipath TCP has been used for several use cases. On smartphones, MPTCP enables seamless handovers between cellular and Wi-Fi networks while preserving established connections. This use-case is what pushed Apple to use MPTCP since 2013 in multiple applications [2]. On dual-stack hosts, Multipath TCP enables the TCP connection to automatically use the best performing path, either IPv4 or IPv6. If one path fails, MPTCP automatically uses the other path. To benefit from MPTCP, both the client and the server have to support it. Multipath TCP is a backward-compatible TCP extension that is enabled by default on recent Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath TCP is included in the Linux kernel since version 5.6 [3]. To use it on Linux, an application must explicitly enable it when creating the socket. No need to change anything else in the application. Adding the possibility to create MPTCP sockets would thus be a really fine addition to httpd, by allowing clients to make use of their different interfaces. This patch introduces the possibility to listen with MPTCP sockets. Note however that these changes are only available on Linux, as IPPROTO_MPTCP is Linux specific for the time being. To do so, we extended the Listen directive to include a \"multipathtcp\" option, allowing to create MPTCP sockets instead of regular TCP ones: Listen 80 options=multipathtcp We then store this information in flags for the listen directive and create sockets appropriately according to this value. Link: https://www.rfc-editor.org/rfc/rfc8684.html [1] Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2] Link: https://www.mptcp.dev [3] Submitted by: Aperence <anthony.doeraene hotmail.com> Github: closes #476
mod_proxy: Fix selection of ProxyPassMatch workers with host/port substitution. PR 69233. With "ProxyPassMatch ^/([^/]+)/(.*)$ https://$1/$2", ap_proxy_get_worker_ex() should not consider the length of scheme://host part of the given URL because of the globbing match on the host part. Fix it by setting worker->s>is_host_matchable when creating a worker with host substitution and avoiding the min_match check in worker_matches() in this case.
mod_proxy: Allow for empty UDS URL hostname in ProxyPass workers too. Using "unix:/udspath|scheme:" or "unix:/udspath|scheme://" for a ProxyPass URL does not work currently, while it works for SetHandler "proxy:unix:...".
*) mod_proxy: Avoid AH01059 parsing error for SetHandler "unix:" URLs in <Location> (incomplete fix in 2.4.62). PR 69160. When SetHandler "unix:..." is used in a <Location "/path"> block, the path gets appended (including $DOCUMENT_ROOT somehow) to r->filename hence the current checks in fixup_uds_filename() to add "localhost" when missing don't work. Fix them.
Follow up to r1919023: fix compilation.
mod_proxy: Don't mangle r->filename when ap_proxy_canon_netloc() fails. ap_proxy_canon_netloc() called from the canon_handler hooks modifies its given url in place, hence &r->filename[6] passed from ap_proxy_canon_url(). This is not an issue if the canon_handler hook succeeds (or declines) since r->filename is usually completely rewritten finally, but on failure it gets truncated. Avoid this by passing a copy of r->filename from the start, the proxy *url and r->filename don't need to point to the same data. * proxy/proxy_util.c(ap_proxy_canon_url): Pass a copy of r->filename to the canon_handler hooks.
mod_proxy: Fix canonicalisation and FCGI env (PATH_INFO, SCRIPT_NAME) for
"balancer:" URLs set via SetHandler, also allowing for "unix:"
sockets with BalancerMember(s). PR 69168.
* modules/proxy/proxy_util.h, modules/proxy/proxy_util.c:
Move proxy_interpolate() from mod_proxy.c to ap_proxy_interpolate(),
exported locally only (non public).
Move proxy_fixup() from mod_proxy.c to ap_proxy_canon_url(), exported
locally only too (non public).
Rollback ap_proxy_fixup_uds_filename() to a local fixup_uds_filename()
usable from proxy_util.c only. The public function will be removed in
a following commit.
* modules/proxy/mod_proxy.h:
Note that ap_proxy_fixup_uds_filename() is deprecated.
* modules/proxy/mod_proxy.c:
Just use ap_proxy_canon_url() from proxy_fixup() and proxy_handler()
for SetHandler URLs.
* modules/proxy/mod_proxy_balancer.c:
Do not canonicalize the path from proxy_balancer_canon() anymore but
rather from balancer_fixup() where the balancer URL is rewritten to
the BalancerMember URL.
Follow up to r1919015: fix compilation.
mod_proxy: Avoid AH01059 parsing error for SetHandler "unix:" URLs. PR 69160 The hostname part of the URL is not mandated for UDS though the canon_handler hooks will require it, so add "localhost" if it's missing (won't be used anyway for an AF_UNIX socket). This can trigger with SetHandler "unix:" URLs which are now also fixed up.
mod_proxy: follow up to r1918626: Simplify ap_proxy_fixup_uds_filename() and callers.
mod_proxy: Fixup UDS filename for mod_proxy called through r->handler. * modules/proxy/proxy_util.c: Export ap_proxy_fixup_uds_filename() from fix_uds_filename. Call it from ap_proxy_pre_request() even for rewritten balancer workers. * modules/proxy/mod_proxy.h: Declare ap_proxy_fixup_uds_filename() * modules/proxy/mod_proxy.c: Fixup UDS filename from r->handler in proxy_handler(). * include/ap_mmn.h: Bump MMN minor for ap_proxy_fixup_uds_filename()
validate hostname
mod_proxy: Follow up to r1918438: Don't cleanup conn before running the address cleanup. * modules/proxy/proxy_util.c(ap_proxy_determine_address): Move conn_cleanup() after apr_pool_cleanup_run().
mod_proxy: Follow up to r1918412 and r1918429: Special case WIN32/OS2 only. apr_socket_connect() on unixes does copy the passed in *addr, so limit the liefetime workaround to Windows and OS/2 only (which don't). * modules/proxy/proxy_util.c(ap_proxy_determine_address): #ifdef the relevant code for WIN32/OS2 only, and improve comment.
mod_proxy: Follow up to r1918412: Find if one of the conn->addr[->next] is alive. ap_proxy_connect_backend() will use the first conn->addr[->next] that works, so the current address alive can be any of them. * modules/proxy/proxy_util.c(ap_proxy_determine_address): Loop for all conn->addr[->next] to determine if addr_alive.
mod_proxy: Keep connection alive on addressTTL expiry if the DNS didn't change. * modules/proxy/proxy_util.c(address_cleanup): Rename to conn_cleanup() since it also closes the socket, and run socket_cleanup() first to avoid dangling conn->sock->remote_addr. * modules/proxy/proxy_util.c(ap_proxy_determine_address): Compare the new address with the old one and keep the socket alive if it did not change. Follow up to r1918410.
mod_proxy: Fix DNS requests and connections closed before the configured addressTTL. BZ 69126 * modules/proxy/proxy_util.c(ap_proxy_determine_address): Fix shared expiry compare-and-swap loop.
* modules/proxy/proxy_util.c (worker_address_resolve): Fix debugging output when dumping multiple resolved addresses.
Don't reset the balancer when we create a new child and don't change shared memory the load balancer is not using.
mod_proxy: Follow up to r1912245: ap_proxy_ prefix for extern functions. Even if they are not part of the API (not in mod_proxy.h) hence requires no MMN bump, {get,set,increment_,decrement_}busy_count() being AP_PROXY_DECLARE()d could name-collide with a third-party module's functions. Rename them using the ap_proxy_ prefix, with an underscore after the verb for for all of them too (for consistency), that is: ap_proxy_{get,set,increment,decrement}_busy_count()
mod_proxy: Follow up to r1912459: Fix reuse of forward_info. Use the correct fwd_pool for allocating the forward_info when the connection is reusable as spotted by Rüdiger. Do not reuse conn->forward if the ->proxy_auth changed.
Fix format string type check
mod_proxy: Simplify ap_proxy_get_worker_ex() Factorize duplicated code in the balancer and non-balancer cases by adding a new worker_matches() helper. No functional change intended.
mod_proxy: Add ap_proxy_worker_get_name() and deprecate ap_proxy_worker_name(). The latter requires a pool and returns a non constant string although it may return worker shared data. By computing the worker "UDS" name at init time we can return a constant name in any case with no need for a pool, that's the new ap_proxy_worker_get_name().
mod_proxy: Consistently close the socket on failure to reuse the connection. proxy_connection_create() and ap_proxy_connect_backend() sometimes close the connection on failure, sometimes not. Always close it.
mod_proxy: Handle backend address renewal with address_ttl= parameter.
Define a new proxy_address struct holding the current/latest sockaddr in use
by each proxy worker and conn. Since backend addresses can be updated when
their TTL expires and while connections are being processed, each address is
refcounted and freed only when the last worker (or conn) using it grabs the
new one.
The lifetime of the addresses is handled at a single place by the new
ap_proxy_determine_address() function. It guarantees to bind the current/latest
backend address to the passed in conn (or do nothing if it's up to date already).
The function is called indirectly by ap_proxy_determine_connection() for the
proxy modules that use it, or directly by mod_proxy_ftp and mod_proxy_hcheck.
It also is called eventually by ap_proxy_connect_backend() when connect()ing all
the current addresses fails, to check (PROXY_DETERMINE_ADDRESS_CHECK) if some
new addrs are available.
This commit is also a rework of the lifetime of conn->addr, conn->hostname
and conn->forward, using the conn->uds_pool and conn->fwd_pool for the cases
where the backend is connected through a UDS socket and a remote CONNECT proxy
respectively.
* include/ap_mmn.h:
Minor bump for new function/fields.
* modules/proxy/mod_proxy.h (struct proxy_address,
ap_proxy_determine_addresss()):
Declare ap_proxy_determine_addresss() and opaque struct proxy_address,
new fields to structs proxy_conn_rec/proxy_worker_shared/proxy_worker.
* modules/proxy/mod_proxy.c (set_worker_param):
Parse/set the new worker->address_ttl parameter.
* modules/proxy/proxy_util.c (proxy_util_register_hooks(),
ap_proxy_initialize_worker(),
ap_proxy_connection_reusable(),
ap_proxyerror(), proxyerror_core(),
init_conn_pool(), make_conn_subpool(),
connection_make(), connection_cleanup(),
connection_constructor()):
Initialize *proxy_start_time in proxy_util_register_hooks() as the epoch
from which expiration times are relative (i.e. seconds stored in an uint32_t
for atomic changes).
Make sure worker->s->is_address_reusable and worker->s->disablereuse are
consistant in ap_proxy_initialize_worker(), thus no need to check for both
in ap_proxy_connection_reusable().
New proxyerror_core() helper taking an apr_status_t to log, wrap in
ap_proxyerror().
New make_conn_subpool() to create worker->cp->{pool,dns} with their own
allocator.
New connection_make() helper to factorize code in connection_cleanup() and
connection_constructor().
* modules/proxy/proxy_util.c (proxy_address_inc(), proxy_address_dec(),
proxy_address_cleanup(), proxy_address_set_expired(),
worker_address_get(), worker_address_set(),
worker_address_resolve(), proxy_addrs_equal(),
ap_proxy_determine_address(),
ap_proxy_determine_connection(),
ap_proxy_connect_backend()):
Implement ap_proxy_determine_address() using the above helpers for atomic changes,
and call it from ap_proxy_determine_connection() and ap_proxy_connect_backend().
* modules/proxy/mod_proxy_ftp.c (proxy_ftp_handler):
Use ap_proxy_determine_address() and use the returned backend->addr.
* modules/proxy/mod_proxy_hcheck.c (hc_determine_connection, hc_get_backend,
hc_init_worker, hc_watchdog_callback):
Use ap_proxy_determine_address() in hc_determine_connection() and call the
latter from hc_get_backend(), replace hc_init_worker() by hc_init_baton()
which now calls hc_get_hcworker() and hc_get_backend() to resolve the first
address at init time.
* modules/proxy/mod_proxy_http.c (proxy_http_handler):
Use backend->addr and ->hostname instead of worker->cp->addr and
worker->s->hostname_ex respectively.
* modules/proxy/mod_proxy_ajp.c (ap_proxy_ajp_request):
Use backend->addr and ->hostname instead of worker->cp->addr and
worker->s->hostname_ex respectively.
Closes #367
Arrange the bybusyness logic and prevent bad busy values this closes #383
proxy_util: revert timeout selection on a proxy tunnel back to take the large of client/origin values. Add websocket test to verify that longer proxypass timeout is in effect.
proxy: in proxy tunnels, use the smaller timeout value of
client and origin as timeout for polling the tunnel.
*) mod_http2: added support for bootstrapping WebSockets via HTTP/2, as
described in RFC 8441. A new directive 'H2WebSockets on|off' has been
added. The feature is by default not enabled.
As also discussed in the manual, this feature should work for setups
using "ProxyPass backend-url upgrade=websocket" without further changes.
Special server modules for WebSockets will have to be adapted,
most likely, as the handling if IO events is different with HTTP/2.
HTTP/2 WebSockets are supported on platforms with native pipes. This
excludes Windows.
* In the reverse proxy case when we only want to keep encoded slashes untouched we can have decoded '%''s in the URI that got sent to us in the original URL as %25. Don't error out in this case but just fall through and have them encoded to %25 when forwarding to the backend. PR: 66580
* If we fail to connect to all looked up IP's from the worker lookup cache it might be caused by a change on DNS side. Try another DNS lookup in this case and in case this causes a successful connection trigger a refresh of the worker lookup cache.
* Rephrase comment [skip ci]
* Clear the dns_pool before to avoid a memory leak in case we did the lookup again.
* For retriggering a DNS lookup worker->cp->addr should be set to NULL and thus we need to avoid a race that worker->cp->addr switches to NULL after we checked it to be non NULL but before we assign it to conn->addr in an else tree which would leave conn->addr to NULL and likely cause a segfault later.
* Whitespace fixes. No functional change.
Do not double encode encoded slashes
In case that AllowEncodedSlashes is set to NoDecode do not double encode
encoded slashes in the URL sent by the reverse proxy to the backend.
* include/ap_mmn.h: Document the addition of ap_proxy_canonenc_ex to the API.
* modules/proxy/mod_proxy.h: Declare ap_proxy_canonenc_ex and define flag
values.
* modules/proxy/proxy_util.c: Implement ap_proxy_canonenc_ex by modifying
ap_proxy_canonenc accordingly and reimplement ap_proxy_canonenc to
use ap_proxy_canonenc_ex with the appropriate flag.
* modules/http2/mod_proxy_http2.c, modules/proxy/mod_proxy_*.c: Set the
correct flag based on the AllowEncodedSlashes configuration and use
ap_proxy_canonenc_ex instead of ap_proxy_canonenc.
* Optimize code: We know that we already have a valid encoding string, because we just decoded it. Hence just copy what we have instead of redoing it.
mod_proxy: Ignore (and warn about) enablereuse=on for ProxyPassMatch when
some dollar substitution (backreference) happens in the hostname
or port part of the URL.
Address or connection reuse can't work when the autority part of the URL is
dynamic (single origin server[:port] handled/assumed in the reslist). Detect
such cases and unset worker->s->is_address_reusable to disable reuse regardless
of enablereuse/disablereuse.
* modules/proxy/proxy_util.c(ap_proxy_define_worker_ex):
Lookup for $n substitution in the hostname[:port] when parsing the URL and
if present, set worker->->is_address_reusable=0 / worker->s->disablereuse=1.
* modules/proxy/proxy_util.c(ap_proxy_initialize_worker):
Don't overwrite worker->s->is_address_reusable from enablereuse/disablereuse
parameters, and set both consistently.
* docs/manual/mod/mod_proxy.xml:
Add ProxyPassMatch compatibility note about key=value parameters handled with
$n substitutions since 2.4.47.
Document the specificities of enablereuse/disablereuse w.r.t. $n subsitutions
in the different part of the URL.
Axe the note about unparsable URLs when the $n substitution happens in the
port, this has been addressed in 2.4.47 too (and works now).
use a liberal default limit for LimitRequestBody of 1GB
mod_proxy: Follow up to r1901485: Don't shadow upper 'val' variable.
mod_proxy: Let fixup hooks know about the Host header (and eventually overwrite it). If proxy_run_fixups() sets a Host header there will be two ones sent to the origin server. Instead, let the hooks know about the Host by setting it in the r->headers_in passed to proxy_run_fixups(), and use the actual value afterwards. Note: if proxy_run_fixups() unsets the Host we'll keep ours. Suggested by: rpluem
mod_proxy: ap_proxy_create_hdrbrgd() to clear hop-by-hop first and fixup last. So that ap_proxy_clear_connection() runs on the original headers only and proxy_run_fixups() on the final ones.
mod_proxy: Align ap_proxy_create_hdrbrgd() with 2.4.x's. In 2.4.x, the copy of r->headers_in is left in r->headers_in for the whole function, while the original r->headers_in are restored at the end. This is simpler and avoids the r->headers_in <=> saved_headers_in danse when calling a function that modifies r->headers_in in place. Align with 2.4.x, no functional change.
mod_proxy_http: Follow up to r1901420: consistent 100-continue checks. Let proxy_http_handler() tell ap_proxy_create_hdrbrgd() whether to add or preserve Expect header or not, through the "proxy-100-continue" note.
mod_proxy_http: Avoid 417 responses for non forwardable 100-continue. PR 65666. Stop returning 417 when mod_proxy has to forward an HTTP/1.1 request with both "Expect: 100-continue" and "force-proxy-request-1.0" set, mod_proxy can instead handle the 100-continue by itself before forwarding the request, like in the "Proxy100Continue Off" case. Note that this does not change the behaviour of httpd receiving an HTTP/1.0 request with an Expect header, ap_check_request_header() will still correctly return 417 in this case.
*) mod_proxy: Add backend port to log messages to
ease identification of involved service.
*) core: add ap_h1_append_header() for single header values.
*) mod_proxy: use of new ap_h1_header(s) functions for
formatting HTTP/1.1 requests.
*) core/mod_http/mod_http2:
- adds new meta bucket types REQUEST, RESPONSE and HEADERS to the API.
- adds a new method for setting standard response headers Date and Server
- adds helper methods for formatting parts of HTTP/1.x, like headers and
end chunks for use in non-core parts of the server, e.g. mod_proxy
- splits the HTTP_IN filter into a "generic HTTP" and "specific HTTP/1.x"
filter. The latter one named HTTP1_BODY_IN.
- Uses HTTP1_BODY_IN only for requests with HTTP version <= 1.1
- Removes the chunked input simulation from mod_http2
- adds body_indeterminate flag to request_rec that indicates that a request
body may be present and needs to be read/discarded. This replaces logic
that thinks without Content-Length and Transfer-Encoding, no request
body can exist.
Remove some APR 0.x stuff. The minimum supported version is 1.3 in 2.4.x
* Change the logic to choose the maximum of both timeouts (front end socket, backend socket) instead of the minimum as backend timeouts can be configured more selectively (per worker if needed) as front end timeouts and typically the backend timeouts reflect the application requirements better. PR: 65886
mod_proxy: Axe useless setting of *balancer to NULL in ap_proxy_pre_request() It's already NULL per the enclosing "if" test. Also, while at a non-functional change, break a close long log line to <80 col.
mod_proxy: follow up to r1895921: Don't prevent forwarding URIs w/ no hostname. r1895921 changed proxy_detect() to disable forward proxying for URIs with no hostname which is wrong, there might exist a third-party proxy module handling the "urn:" scheme for instance (thanks Roy for the catch!). For this to work, we also need to leave the forward proxied URI alone in ap_proxy_pre_request() with no UDS special case or alike, a proxy module can then catch (or not) the original URI as expected.
mod_proxy: Detect unix: scheme syntax errors at load time.
* modules/proxy/mod_proxy.c(add_pass, add_member, set_proxy_param,
proxysection):
Check return value of ap_proxy_de_socketfy().
* modules/proxy/proxy_util.c(ap_proxy_get_worker_ex):
Check return value of ap_proxy_de_socketfy().
mod_proxy: SetEnv proxy-nohalfclose to disable half-close tunneling. PR 65662. Some connect/wstunnel protocols might want half-close forwarding while some might not, let's provide an r->subprocess_env opt-out.
mod_proxy_connect: Honor the smallest of the backend or client timeout. It seems that mod_proxy_connect has never applied any timeout in its tunneling loop. Address this by setting a default timeout in ap_proxy_tunnel_create() since mod_proxy_connect does not overwrite tunnel->timeout (while proxy_http and proxy_wstunnel do). This default timeout is set to the smallest of the backend side or the client side timeout. PR 65631, 65662, 65689
mod_proxy: Add tunnel_forward hook. * modules/proxy/mod_proxy.h, modules/proxy/mod_proxy.c: Declare/implement the hook. * modules/proxy/proxy_util.c(proxy_transfer): Run tunnel_forward hooks when called by the tunneling loop. Simpler input/output brigade cleanup on exit.
mod_proxy: Handle ap_proxy_buckets_lifetime_transform() errors.
* modules/proxy/mod_proxy.h,modules/proxy/proxy_util.c:
Add ap_proxy_fill_error_brigade() to factorize proxy error handling
on the client connection side.
* modules/proxy/mod_proxy_{http,ajp,uwsgi}.c:
Use ap_proxy_fill_error_brigade() where needed, including when an
empty brigade is returned on the backend side or when calling
ap_proxy_buckets_lifetime_transform fails.
mod_proxy: Restore ap_proxy_transfer_between_connections().
* modules/proxy/mod_proxy.h:
Restore the interface of ap_proxy_transfer_between_connections() back to
before MMN 20210506.0.
Add ap_proxy_tunnel_conn_bytes_in() and ap_proxy_tunnel_conn_bytes_out().
New proxy_tunnel_conn_t typedef'ed from opaque struct proxy_tunnel_conn.
* modules/proxy/mod_proxy.h(ap_proxy_tunnel_conn_get_read,
ap_proxy_tunnel_conn_get_transferred):
Axed/replaced by ap_proxy_tunnel_conn_bytes_in() and
ap_proxy_tunnel_conn_bytes_out().
* modules/proxy/proxy_util.c(struct proxy_tunnel_conn):
Replace "exchanged" by "bytes_in" and "bytes_out".
* modules/proxy/proxy_util.c(proxy_transfer):
New helper implementing ap_proxy_transfer_between_connections() and
returning both &bytes_in and &bytes_out.
* modules/proxy/proxy_util.c(ap_proxy_transfer_between_connections):
Now calls proxy_transfer().
* modules/proxy/proxy_util.c(ap_proxy_tunnel_conn_bytes_in,
ap_proxy_tunnel_conn_bytes_out):
Return tc->bytes_in and tc->bytes_out respectively.
* modules/proxy/proxy_util.c(proxy_tunnel_forward):
Use proxy_transfer() which updates in->bytes_in, &out->bytes_out.
tunnel->replied will be updated in proxy_tunnel_run().
* modules/proxy/proxy_util.c(proxy_tunnel_forward):
Fall through the "done" label in any case to set tunnel->replied based
on tunnel->client->bytes_out > 0.
* modules/proxy/mod_proxy_http.c(ap_proxy_http_process_response):
Use ap_proxy_tunnel_conn_bytes_{in,out}() for worker->s->{read,transferred}
accounting.
* modules/proxy/mod_proxy_http.c(proxy_http_async_finish):
Update worker->s->{read,transferred} when async too.
mod_proxy: typedef struct proxy_tunnel_conn proxy_tunnel_conn_t.
mod_proxy: Handle UDS URIs with empty hostname as if they had no hostname. It was reported to me (privately) that r1893101 broke existing settings like: SetHandler "proxy:unix:///path/to/uds.sock|fcgi://localhost/" RFC 3986 (section 3.2.2) says that: If the URI scheme defines a default for host, then that default applies when the host subcomponent is undefined or when the registered name is empty (zero length). For example, the "file" URI scheme is defined so that no authority, an empty host, and "localhost" all mean the end-user's machine, whereas the "http" scheme considers a missing authority or empty host invalid. Let's consider that the "unix" scheme is closer to the "file" scheme than the "http" one, and accept "unix:///path/to/uds.sock" as a valid URI.
core: Add ap_create_connection() to create a server or client/proxy connection. c->outgoing shouldn't be set by mod_ssl, ap_create_connection() allows that and this commit also replaces all the calls to ap_run_create_connection() in mod_proxy modules (not in the MPMs which create incoming connections only). * include/http_connection.h, server/connection.c: Declare and implement ap_create_connection(). * modules/proxy/proxy_util.c, modules/proxy/mod_proxy_connect.c, modules/proxy/mod_proxy_ftp.c: Use ap_create_connection() instead of ap_run_create_connection(), and don't provide a connection_id a scoreboard handle for outgoing connection. * server/log.c(do_errorlog_default): Use c->outgoing instead of c->sbh to determine if it's a "client" or "remote" connection.
mod_proxy: Follow up to r1892986: APLOGNO() Stefan get out of this body! :)
mod_proxy: Follow up to r1892814. * modules/proxy/proxy_util.c(fix_uds_filename): Sanity checks on the configured UDS path, fail with 500 if invalid since continuing through proxy processing wouldn't work as expected.
Fix a compilation error in the configuration where this is compiled. Unless I missed something, it is broken since r1516930 (Aug 23 2013). So no-one has compiled this for more than 8 years. Sounds like dead code.
mod_proxy: Follow up to r1892740. Really remove the old ap_filter_input_pending() handling forgotten by r1892740.
mod_proxy: Faster unix socket path parsing in the "proxy:" URL. The actual r->filename format is "[proxy:]unix:path|url" for UDS, no need to strstr(,"unix:") since it's at the start of the string.
mod_proxy: Fix potential tunneling infinite loop and spurious timeout.
PRs 65521 and 65519.
* modules/proxy/proxy_util.c(ap_proxy_tunnel_run):
Avoid an infinite loop by shutting down the connection for write when poll()
returns POLLHUP and read is already down. PR 65521.
* modules/proxy/proxy_util.c(ap_proxy_tunnel_run):
When write completion is finished don't check for ap_filter_input_pending()
before proxy_tunnel_forward() to flush input data, this is a nonblocking read
already which will do the same thing implicitely. ap_filter_input_pending()
is broken in 2.4.x without the whole pending data mechanism (not backported
yet), so let's align here. PR 65519.
core: follow up to r1891148: WC bucket defaulting to FLUSH bucket. Define WC bucket semantics as: /** * @brief Write Completion (WC) bucket * * A WC bucket is a FLUSH bucket with special ->data == &ap_bucket_wc_data, * still both AP_BUCKET_IS_WC() and APR_BUCKET_IS_FLUSH() hold for them so * they have the same semantics for most filters, namely: * Everything produced before shall be passed to the next filter, including * the WC/FLUSH bucket itself. * The distinction between WC and FLUSH buckets is only for filters that care * about write completion (calling ap_filter_reinstate_brigade() with non-NULL * flush_upto), those can setaside WC buckets and the preceding data provided * they have first determined that the next filter(s) have pending data * already, usually by calling ap_filter_should_yield(f->next). */ The only filters that care about write completion for now are ap_core_output_filter() and ssl_io_filter_output(), which try to fill in the pipe as much as possible, using ap_filter_reinstate_brigade(&flush_upto) to determine whether they should flush (blocking) or setaside their remaining data. So ap_filter_reinstate_brigade() is made to not treat WC as FLUSH buckets and keep the above filters working as before (and correctly w.r.t. above WC bucket semantics). * include/ap_mmn.h, include/util_filter.h: Axe specific ap_bucket_type_wc and define global &ap_bucket_wc_data address to mark WC buckets checked by AP_BUCKET_IS_WC(). * server/util_filter.c (ap_filter_reinstate_brigade): Don't treat WC buckets as FLUSH buckets.
Follow-up to r1891477. Axe a few lines of code and document a potential optimization
mod_proxy: Fix icomplete initialization of BalancerMember(s) from the manager. Clear the workers created in ap_proxy_sync_balancer(), notably ->local_status for below ap_proxy_initialize_worker() to initialize all the child structures like ->cp and ->cp->reslist, avoiding a possible crash when the workers are used at runtime.
mod_proxy: Avoid confusion of prefix/regex matching workers at loading. PR 65429. ap_proxy_get_worker() needs to know whether it should lookup for prefix or match or both matching workers, depending on the context. For instance <Proxy[Match]> or ProxyPass[Match] directives need to lookup for an existing worker with the same type as the directive (*Match or not), because they will define one with that matching type if none exists. On the contrary, "ProxySet <url>" at load time or ap_proxy_pre_request() at run time need to find a worker matching an url whether it's by prefix or by regex. So this commit adds ap_proxy_get_worker_ex() which takes a bitmask for the matching type and calls it appropriately where needed. For consistency, ap_proxy_define_worker_ex() is also added, using the same bitmask flags, deprecating ap_proxy_define_match_worker(). Follow up to r1891206.
mod_proxy: Follow up to r1891206: fix UDS scheme.
mod_proxy: Fix possible reuse/merging of Proxy(Pass)Match workers. PR 65419. We can't truncate ProxyMatch's worker name/url to the first '$' substitution without possibly colliding with other workers. This also makes the matching done at runtime by ap_proxy_strcmp_ematch() completely pointless. To fix this and still address r1878467 (i.e. make http://host:port$1 a "valid" URL), we need to remove '$' substitutions from the :port part of the URL only since it's allowed anywhere else by apr_uri_parse(). So let's strip them before apr_uri_parse() and prepend them back in the path before apr_uri_unparse() to restore the original URL. Non-matchable workers are not concerned so ap_proxy_define_worker() is made a local helper (w/o the ap_ prefix) which takes "matchable" as argument and can then be called by both ap_proxy_define_[match_]worker() functions.
core: Write Completion (WC) bucket type. A WC bucket is meant to prevent buffering/coalescing filters from retaining data, but unlike a FLUSH bucket it won't cause the core output filter to block trying to flush anything before. It can be passed by async handlers which want to never block, followed by ap_filter_should_yield() to check for pending data and eventually suspend processing until MPM/asynchronous write completion finishes. In this commit it's used that way by the tunneling loop of mod_proxy to prevent SSL coaslescing. gh: closes #200
Allow the tunnelled connections to report the read and trasnfered to the back-end worker.
* modules/proxy/proxy_util.c (ap_proxy_define_balancer): Fix leak in error path in the do_malloc case, caught by covscan.
mod_proxy_wstunnel: leave Upgrade requests handling to mod_proxy_http.
Let mod_proxy_http's canon and scheme handlers accept "ws[s]:" schemes so that
mod_proxy_wstunnel can decline requests when mod_proxy_http is loaded.
* modules/proxy/{mod_proxy.h,proxy_util.c} (ap_proxy_worker_can_upgrade):
Add a "dflt" argument to ap_proxy_worker_can_upgrade() which, if not NULL,
is matched when no worker upgrade= parameter is configured. This allows to
handle the default "Upgrade: websocket" case for "ws[s]:" schemes.
* modules/proxy/mod_proxy_http.c (proxy_http_canon, proxy_http_handler):
Add and use the new get_url_scheme() helper to parse URL schemes handled by
mod_proxy_http and use it in canon and scheme handlers. This helper now
accepts ws[s] schemes.
* modules/proxy/mod_proxy_wstunnel.c (proxy_wstunnel_post_config):
New post_config hook to detect whether mod_proxy_http is loaded and set
global fallback_to_mod_proxy_http flag in this case.
* modules/proxy/mod_proxy_wstunnel.c (proxy_wstunnel_check_trans,
proxy_wstunnel_canon,
proxy_wstunnel_handler):
These hooks now early return DECLINED if fallback_to_mod_proxy_http is set.
Revert r1480058, -1'ed on dev@ and STATUS. Never backported (and never will supposedly), while often creating merge conflicts. See https://lists.apache.org/thread.html/be0e7bdc3510fddd2dd80accece44917eba361ef4fcc713dd0f7f7fa%401367999236%40%3Cdev.httpd.apache.org%3E and https://lists.apache.org/thread.html/6e63271b308a2723285d288857318e7bb51b6756690514d9bc75a71b%401371148914%40%3Ccvs.httpd.apache.org%3E
mod_proxy: provide prefetching and spooling mechanisms to all proxy modules. Export ap_proxy_prefetch_input(), ap_proxy_spool_input() and ap_proxy_read_input() from mod_proxy_http to mod_proxy.h/proxy_util.c so that they are usable by all proxy modules. mod_proxy_fcgi will use them in a following commit.
mod_proxy: pconf vs pchild consistency, and correctness in ONE_PROCESS mode.
Consistently use pconf for ap_proxy_define_{worker,balancer}() and pchild for
ap_proxu_initialize_{worker,balancer}() in mod_proxy [child_]init code.
pchild is needed in _initialize() for mutexes/shms' child_init and cleanup,
and to avoid a crash on shutdown (i.e. ap_terminate) in ONE_PROCESS mode,
where worker->cp->pool is destroyed twice, let's register conn_pool_cleanup()
as a pre_cleanup of pchild.
mod_proxy: Add support for an optional third argument to ProxyRemote* to configure the Basic auth credentials to send to the remote proxy. (Note that credentials are always sent w/o waiting for a challenge as with proxy-chain-auth, and only Basic is supported - both of which are not exactly ideal - but better than nothing.) * modules/proxy/mod_proxy.h (struct proxy_remote): Add creds field. * modules/proxy/mod_proxy.c (proxy_handler): Pass forward proxy credentials via r->notes. (add_proxy): Take credentials and base64-encode into ->creds field if passed. (add_proxy_noregex, add_proxy_regex): Take optional creds argument. * modules/proxy/proxy_util.c (ap_proxy_determine_connection): Use proxy credentials from r->notes if available. (ap_proxy_create_hdrbrgd): Set Proxy-Authorization header from credentials in r->notes if present. PR: 37355 Github: closes #135
mod_proxy: follow up to r1879401: call filters on tunnel POLLERR. proxy_util.c: Set POLLERR in reqevents for pollset providers that require it to detect socket errors (like select() based one). Call filters to read/write on POLLERR socket event, so that they know about the error by experiencing the failure. If no POLLIN|POLLOUT is returned with POLLERR (depending on the system or pollset provider), go with the requested read or write event handling. Restore ap_proxy_transfer_between_connections() so that it always tries to read first (i.e. move yielding conditions afterward). Add proxy_tunnel_forward() helper that calls transfer_between_connections() and handles errors pollset updates. Call proxy_tunnel_forward() when write completion finishes and there are pending input data. mod_proxy.h: Add read_buf_size to proxy_tunnel_rec (trunk only, no MMN minor bump).
mod_ssl: release coalesced data when called from ap_filter_output_pending(). The purpose of ap_filter_output_pending() is to flush pending data, so ssl_io_filter_coalesce() should honor that. This allows mod_proxy to not care about mod_ssl coalescing filters when tunneling between connections.
mod_proxy: reindent ap_proxy_transfer_between_connections() after r1879401. No functional change.
mod_proxy: improved and reentrant tunneling loop.
modules/proxy/mod_proxy.h:
Rename AP_PROXY_TRANSFER_SHOULD_YIELD to AP_PROXY_TRANSFER_YIELD_PENDING
and add AP_PROXY_TRANSFER_YIELD_MAX_READS.
modules/proxy/mod_proxy_http.c:
modules/proxy/mod_proxy_wstunnel.c:
Removing of reqtimeout filter is now handled by ap_proxy_tunnel_create().
modules/proxy/proxy_util.c:
ap_proxy_transfer_between_connections():
Reorganize loop to break out early.
When AP_PROXY_TRANSFER_YIELD_PENDING, if !ap_filter_should_yield() we
still need to run and check ap_filter_output_pending() since it may
release pending data.
When AP_PROXY_TRANSFER_YIELD_MAX_READS, stop the loop after too much
reads (PROXY_TRANSFER_MAX_READS = 10000) to release the thread and
give the caller a chance to schedule the other direction.
Don't return APR_INCOMPLETE when it comes from an incomplete body
detected by ap_http_filter().
ap_proxy_tunnel_create():
Start with POLLOUT on both directions so that any pending output data
is flushed first.
ap_proxy_tunnel_run():
Remove re-init/clear of the pollset for each call so that the function
is reentrant.
Handle POLLOUT before POLLIN so that we can read in the same pass once
all buffered output data are flushed, using ap_filter_input_pending()
to drain buffered input data.
This is preparatory patch for async websocket tunneling is mod_proxy_http.
ap_proxy_define_match_worker: disable connection reuse by default. To avoid compat issues with dns/connection reuse now that a worker with dollar substitution can be elected.
ap_proxy_define_match_worker: don't copy the url unnecessarily. And save a few cycles, when the duplication is needed, by not copying the ignored part.
make sure the $n of the regular expressions is not included the name of the worker. for example, the example: ProxyPassMatch "^(/.*\.gif)$" "http://backend.example.com:8000$1" was giving: AH00526: Syntax error on line nnn of bla/conf/httpd.conf: ProxyPass Unable to parse URL: http://backend.example.com:8000$1
mod_proxy: binary search for ProxyErrorOverride status codes. The list can be rather long, speed up runtime by sorting the status codes in error_override_codes and using binary search from ap_proxy_should_override().
mod_proxy_http: handle Upgrade requests and upgraded protocol forwarding.
If the request Upgrade header matches the worker upgrade= parameter and
the backend switches the protocol, do the tunneling in mod_proxy_http.
This allows to keep the protocol to HTTP until the backend really
switches the protocol, and apply usual output filters.
When configured to forward Upgrade mechanism, we want the backend to be
able to announce its Upgrade protocol to the client (e.g. with 426
Upgrade Required response) and thus forward back the Upgrade header that
matches the one(s) configured in the worker upgrade= parameter.
modules/proxy/mod_proxy.h:
modules/proxy/proxy_util.c:
ap_proxy_worker_can_upgrade(): added helper to determine whether a
proxy worker is configured to forward an Upgrade protocol.
include/ap_mmn.h:
Bump MMN minor for ap_proxy_worker_can_upgrade().
modules/proxy/mod_proxy.c:
set_worker_param(): handle worker parameter upgrade=ANY as upgrade=*
(should the "any" protocol scheme be something some day..).
modules/proxy/mod_proxy_wstunnel.c:
proxy_wstunnel_handler(): use ap_proxy_worker_can_upgrade() to match
the Upgrade header. Axe handling of upgrade=NONE, it makes no sense to
Upgrade a connection if the client did not ask for it, nor to configure
mod_proxy_wstunnel to use a worker with upgrade=NONE by the way.
modules/proxy/mod_proxy_http.c:
proxy_http_req_t: add fields force10 (force HTTP/1.0) and upgrade (value
of the Upgrade header sent by the client if it matches the configuration,
NULL otherwise).
proxy_http_handler(): use ap_proxy_worker_can_upgrade() to determine
whether the request is electable for end to end protocol upgrading and set
req->upgrade accordingly.
terminate_headers(): handle Connection and Upgrade headers to send to the
backend, according to req->force10 and req->upgrade set before.
ap_proxy_http_prefetch(): use req->force10 and terminate_headers().
send_continue_body(): added helper to send the body retained for end to
end 100-continue handling.
ap_proxy_http_process_response(): use ap_proxy_worker_can_upgrade() to
match the response Upgrade header and forward it back if it matches the
configured one(s). That is for 101 Switching Protocol obviously but also
any other status code which is not overidden, at the backend wish. If the
protocol is switching, create a proxy tunnel and run it, using the minimal
timeout from the client or backend connection.
Github: closes #125
Add missing pool tags to help debugging.
PR63628: individual status codes for ProxyErrorOverride. Support specifying the http status codes to be considered by ProxyErrorOverride Submitted By: Martin Drößler <mail martindroessler.de> Committed By: covener
Fix spelling errors found by codespell. [skip ci]
Axe an empty line in order to synch with 2.4.x
mod_proxy: Improve tunneling loop. Support half closed connections and pending data draining (for protocols like rsync). PR 61616. When reading on one side goes faster than writing on the other side, the output filters chain may start buffering data and finally block, which will break bidirectional tunneling for some protocols. To avoid this, proxy_tunnel_run() now stops polling/reading until pending data are drained, and recovers appropriately.
mod_proxy: factorize mod_proxy_{connect,wstunnel} tunneling code in proxy_util.
This commit adds struct proxy_tunnel_rec that contains the fields needed for a
poll() loop through the filters chains, plus functions ap_proxy_tunnel_create()
and ap_proxy_tunnel_run() to respectively initialize a tunnel and (re)start it.
Proxy connect and wstunnel modules now make use of this new API to avoid
duplicating logic and code.
Fix pool concurrency problems Create a subpool of the connection pool for worker scoped DNS resolutions. This is needed to avoid race conditions in using the connection pool by multiple threads during ramp up. Recheck after obtaining the lock if we still need to do things or if they were already done by another thread while we were waiting on the lock. * modules/proxy/proxy_util.c: Create a subpool of the connection pool for worker scoped DNS resolutions and use it. * modules/proxy/mod_proxy.h: Define AP_VOLATILIZE_T and add dns_pool to struct proxy_conn_pool. * modules/proxy/mod_proxy_ftp.c: Use dns_pool and consider that worker->cp->addr is volatile in this location of the code. PR: 63503
* Add back logging goodness Add back logging goodness added by covener in r1865938.
* Revert r1865944 and r1865938 Revert r1865944 and r1865938 in order to provide a better patch with less locking and thus contention. lognos PR63503: fix pool concurrency problems in mod_proxy reslist and resolver related calls could concurrently access the same pool. Submitted By: Don Poitras <Don.Poitras sas.com> Committed By: covener
restore use of global mutex under !APR_HAS_THREADS followup to r1852442 which appears to have been too agressive in wrapping blocks with #if APR_HAS_THREADS. With !APR_HAS_THREADS a global mutex is a proc mutex.
lognos
PR63503: fix pool concurrency problems in mod_proxy reslist and resolver related calls could concurrently access the same pool. Submitted By: Don Poitras <Don.Poitras sas.com> Committed By: covener
remove APR_HAS_THREADS check no need to wrap these after r1865936
* modules/proxy/proxy_util.c (ap_proxy_share_balancer): Create the nonce as a pseudo-UUID using the PRNG.
remove request details from error documents (CVE-2019-10092).
* Fix the logic to follow the comment. So far we only forwarded the header if we have NOT used it AND the env variable was set. But if we have not used it we should forward it in any case independent of the env variable setting. This aligns also with the behaviour in ap_proxy_create_hdrbrgd.
Follow up to r1859371: extend to other ap_proxy_connection_create[_ex]() users. This function now now handles SSL reuse as well as the "proxy-request-hostname" note (SNI), so let's also call it unconditionnaly in all proxy modules. On the mod_ssl side, since this note has the lifetime of the connection, don't reset/unset it during handshake (ssl_io_filter_handshake).
Fix for: [Bug 62372] Load balancer byrequests required when bytraffic chosen
mod_proxy/ssl: cleanup per-request SSL configuration for recycled proxy conns. The SSL dir config of proxy/backend connections is stored in r->per_dir_config but those connections have a lifetime independent of the requests they handle. So we need to allow the external ssl_engine_set() function to reset mod_ssl's dir config in between proxy requests, or the first sslconn->dc could be used after free for the next requests. mod_proxy can then reset/reinit the request config when recycling its backend connections. PR 63256.
Make proxy modules compile if APR_HAS_THREADS is not defined.
If ProxyPassReverse is used for reverse mapping of relative redirects, subsequent ProxyPassReverse statements, whether they are relative or absolute, may fail. PR 60408 [Peter Haworth <pmh1wheel gmail.com>]
mod_proxy_http: follow up to r1836588: avoid 100-continue responses from core. When mod_proxy_http handles end-to-end "100 continue", it can't let ap_http_filter() send its own interim response whenever the body is read. So save/restore r->expecting_100 before/after handling the request, and use req->expecting_100 internally (including to restore r->expecting appropriately). While at it, add comments and debug logs about 100 continue handling, and fill in missing APLOGNO()s from r1836588.
* ap_proxy_balancer_get_best_worker cannot be exported and used as an optional function at the same time. So rename ap_proxy_balancer_get_best_worker to proxy_balancer_get_best_worker and make it static which is then used as an optional function and recreate ap_proxy_balancer_get_best_worker as an exported thin wrapper of proxy_balancer_get_best_worker.
* mod_proxy: Remove load order and link dependency between mod_lbmethod_* modules and mod_proxy by providing mod_proxy's ap_proxy_balancer_get_best_worker as an optional function. PR: 62557
Add default schema ports for websockets PR:62480 Lubos Uhliarik <luhliari@redhat.com>
* modules/proxy/proxy_util.c (ap_proxy_share_worker): Skip creating subpool for debugging unless debug-level logging is enabled. No functional change.
In 'ap_proxy_cookie_reverse_map', iterate over each token of the 'Set-Cookie' header field in order to avoid updating the wrong one. This could happen if the header field has something like 'fakepath=foo;path=bar". In this case fakepath would be updated instead of path. We don't need regex anymore in order to parse the field values and 'ap_proxy_strmatch_domain' and 'ap_proxy_strmatch_path' are now useless. (and should be axed IMHO) PR 61560
mod_proxy_balancer: Add hot spare member type and corresponding flag (R). Hot spare members are used as drop-in replacements for unusable workers in the same load balancer set. This differs from hot standbys which are only used when all workers in a set are unusable. PR 61140.
Follow up to r1609680: further simplify/optimize ap_proxy_strcmp_ematch(). While at it, same treatment for its mother ap_strcmp_match().
Follow up to r1609680: simpler/faster ap_proxy_strcmp_ematch(). No functional change.
Set the notice when hostname is too long for legacy proxy modules to info level.
Tone down the message that worker hostname is too long noting it only affects legacy modules not yet using hostname_ex.
mod_proxy: Provide an RFC1035 compliant version of the hostname in the proxy_worker_shared structure. PR62085
too-long worker schemes and/or hostnames are no longer fatal errors
mod_proxy: follow up to r1822849 and r1822879. Fix my maths, thanks Stefan and Rüdiger!
mod_proxy: follow up to r1822849 and r1822878. Does r1822878's "static" APR_RETRIEVE_OPTIONAL_FN work if, say, mod_proxy is builtin but mod_http2 isn't? Not worth taking the risk here since it's not a fast path... Note: if this is an issue, I'm afraid it applies elsewhere too.
mod_proxy: follow up to r1822849. Get the help(er) of mod_http2 to determine how much connections should be handled in the reslist by default (i.e. max_threads).
* Add some some comment why we do not limit hmax any longer
* When mod_http2 is loaded more then ThreadsPerChild backend connections can be useful as mod_http2 has an additional thread pool on top of ThreadsPerChild. But leave the default with ThreadsPerChild.
mod_proxy: fix proxy connection cleanup from an n+2 pool. When connection_destructor() is called after pchild is gone, we can't dereference worker->cp anymore. This happens is debug/one_process mode only, if we exit by calling apr_terminate() or clearing the process pool directly. Fix this by NULL-ing worker->cp in conn_pool_cleanup(), and by registering it as a pre_cleanup.
mod_proxy: allow SSLProxyCheckPeer* usage for all proxy modules. PR 61857. Proposed by: Markus Gausling <markusgausling googlemail.com> Reviewed by: ylavic
loadfactor now decimal
mod_proxy_{ajp,fcgi}: Fix a possible crash when reusing an established
backend connection, happening with LogLevel trace2 or higher configured,
or at any log level with compilers not detected as C99 compliant (e.g.
MSVC on Windows).
Follow up to r1750392: reduce AH03408 level to INFO as suggested by wrowe/jim.
[mod_proxy_]http: follow up to r1750392. Export [ap_]check_pipeline() and use it also for ap_proxy_check_connection(), so that all the necessary checks on the connection are done before reusing it.
Revert r1756064 and r1756060 until fixed (tests framework passes).
[mod_proxy_]http: follow up to r1750392. Export [ap_]check_pipeline() and use it also for ap_proxy_check_connection(). [Reverted by r1756065]
Fix spelling in comments and text files. No functional change. PR 59990
mod_proxy: follow up to r1750392 and r1750474. Restore PROXY_WORKER_IS_USABLE() check in ap_proxy_connect_backend(), we must obviously (un)put backend in error state based on the result of the actual connect(), and don't change it in ap_proxy_check_backend()... APR_SUCCESS return by ap_proxy_check_backend(), i.e. a usable worker and an established connection, is enough for modules to continue w/o calling ap_proxy_connect_backend(), still.
mod_proxy: follow up to r1750392. Avoid double checking the connection in ap_proxy_connect_backend() when ap_proxy_check_backend() says it is up and good to go. This can be done by moving the PROXY_WORKER_IS_USABLE() check in ap_proxy_check_backend(), since it is called by ap_proxy_connect_backend(), and not calling the latter if the former succeeded (for the modules using it).
mod_proxy: we don't need ap_proxy_ssl_connection_cleanup() anymore with ap_proxy_check_backend() used at connection reuse time, so remove its last call and deprecate it.
mod_proxy: use proxy_conn_rec->tmp_bb where appopriate to avoid temporary brigade create/destroy.
mod_proxy_{http,ajp,fcgi}}: don't reuse backend connections with data available
before the request is sent. PR 57832.
ap_proxy_check_backend() can be used before ap_proxy_connect_backend() to try
to read available data (including from the filters), and is called by
ap_proxy_connect_backend() to check the socket state only (as before, still
relevant after ap_proxy_check_backend() due to filter data which may not have
triggered a real socket operation).
mod_proxy: revert r1750301, r1750305 and r1750311. Rework needed.
mod_proxy: follow up to r1750301: APLOGNO(). [Reverted by r1750376]
mod_proxy: follow up to r1750301. s/is_socket_connected/get_socket_connected/ in the !USE_ALTERNATE_IS_CONNECTED case, and add CHANGE + MMN entries. [Reverted by r1750376]
mod_proxy: don't reuse backend connections with data available before the request is sent. PR 57832. [Reverted by r1750376]
When ap_proxy_show_hcmethod has been backported in 2.4.x (r1744951) the function has not been placed at the same position in the code (see r1725306). Move it at the same place in oder to synch 2.4.x and trunk.
Rename ap_casecmpstr[n]() to ap_cstr_casecmp[n](), update with APR doxygen
mod_proxy, mod_ssl: Handle SSLProxy* directives in <Proxy> sections, allowing per backend TLS configuration.
adding default port numbers for h2/h2x proxy schemes, by jchampion
mod_proxy: follow up to r1729826 + r1729847 + r1732986. Don't use magic constants.
Revert r1733054: spurious commit of being-worked mod_reqtimeout changes.
mod_proxy: follow up to r1729826 + r1729847 + r1732986. Don't use magic constants. Reverted by r1733055, re-commited without spurious changes in r1733056.
mod_proxy: follow up to r1729826 + r1729847. Adjust stacked ssl_hostname maximum size.
mod_proxy: add missing APLOGNO()s.
mod_proxy: follow up to r1729826: really copy conn->ssl_hostname.
mod_proxy: Play/restore the TLS-SNI on new backend connections which had to be issued because the remote closed the previous/reusable one during idle (keep-alive) time.
let proxy handler forward ALPN protocol strings for ssl proxy connections
Provide dummy impl of ap_proxy_connect_uds() in case when APR_HAVE_SYS_UN_H is not defined. Just returns APR_ENOTIMPL. This helps symbol export/import handling on Netware and also seems to be more correct, because mod_proxy.h contains ap_proxy_connect_uds() unconditionally.
* Introduce ap_proxy_transfer_between_connections
STOPPED workers are never retried... Disabled can be.
Since these are externs, prepend proxy_
* Fill missing log number
* Introduce ap_proxy_buckets_lifetime_transform as a replacement for proxy_buckets_lifetime_transform in mod_proxy_http. Another future consumer of this will be mod_proxy_wstunnel.
Added many log numbers to log statements that had none. Those were not detected by the coccinelle script.
Expose this in our API
Make balancer manager health-check aware
Oops :)
1st cut of 'simple' tcp check... We reuse various proxy function and so this *could* be more streamlined, but use this to show how the other would work, since we need brigades, SSL/TLS support, etc.
Make aware of new status: Failed Health Check. Store the number of current passes/fails in shm
Abstract out as macro
Use new ap_casecmpstr[n]() functions where appropriate (not exhaustive).
Revert r1715789: will re-commit without spurious functional changes.
Use new ap_casecmpstr[n]() functions where appropriate (not exhaustive). [Reverted by r1715869]
mod_proxy: Fix ProxySourceAddress binding failure with AH00938. PR 56687. Proposed by: Arne de Bruijn <apache arbruijn.dds.nl> Reviewed by: ylavic
mod_proxy: don't recyle backend announced "Connection: close" connections. Failing to do this may lead to a race condition where we send a new request before the backend really closes the connection (or lost SSL-Alert/FIN make us think the connection is still alive, until the retransmission).
* Do a more complete cleanup here. At this point we cannot end up with something useful with the data we created so far.
mod_proxy: follow up to r1678768. The resource is not dereferenceable if connection_destructor() is called from reslist_destroy().
Improve error message (related to PR57311 diagnostic)
mod_proxy: check for the correct pool (lifetime) in connection_cleanup(), according to the associated comment and the value really NULLed in conn_pool_cleanup().
mod_proxy: only cleanup the socket for a connection asked to be closed but whose address can still be reused. This saves unnecessary socket pool destroy and creation at cleanup and reuse time, plus the same initialization of conn->pool's associated data which can be reused in that case.
mod_proxy: use the original (non absolute) form of the request-line's URI for requests embedded in CONNECT payloads used to connect SSL backends via a ProxyRemote forward-proxy. PR 55892. Submitted by: Hendrik Harms <hendrik.harms gmail com> Reviewed by: wrowe, ylavic Committed by: ylavic
* Do not reset the retry timeout if the worker is in error at this stage even if the connection to the backend was successful. It was likely set into error by a different thread / process in parallel e.g. for a timeout or bad status. We should respect this and should not continue with a connection via this worker even if we got one.
mod_proxy_http: don't connect or reuse backend before prefetching request body. The goal is to minimize the delay between this connection is considered alive and the first bytes sent (should the client's link be slow or some input filter retain the data). This is a best effort to prevent the backend from closing (from under us) what it thinks is an idle connection, hence to reduce to the minimum the unavoidable local ap_proxy_is_socket_connected() vs remote keepalive race condition. PR 56541. Also, allow the new subprocess_env variable "proxy-flushall" to prevent any buffering of the request body before it is forwarded to the backend. When set, the prefetch is still done (although non-blocking), so we can still determine Content-Length vs chunked vs spooled (depending on data available with the header or while reading it), and then all brigades are flushed when passed to the backend. PR 37920.
mod_proxy: Don't put non balancer-member workers in error state by default for connection or 500/503 errors, and honor status=+I for any error. PR 48388.
PR 56603: Inappropiate ProxyPassReverse match when interpolated URL is empty string Submitted By: <ajprout hotmail.com> Committed By: covener
Allow SetHandler+UDS+fcgi to take advantage of dedicated workers including opting in to connection reuse and other proxy options (max=, etc). adds 'enablereuse' proxyoption and a minor MMN bump to share proxy_desocketfy outside of mod_proxy.c, which is required to match workers to URLs.
mod_proxy: don't add the default port to the name of proxy workers. PR 57259. ap_proxy_port_of_scheme() knows more default ports than apr_unparse_uri().
elaborate on a TRACE2 message about using the default reverse proxy worker.
Return a match whenever we get to the end of the worker name, regardless of whether there is URL left. ProxyPassMatch had been using the default worker in trunk.
When using EBCDIC encoding, HTTPS through ProxyPass and ProxyRemote doesn't work correctly. PR 57092 Submitted By: Edward Lu Committed By: covener
* Use the correct server name for SNI in case the backend SSL connection itself is established via a proxy server. PR: 57139 Submitted by: Szabolcs Gyurko <szabolcs gyurko.org>
oops... prepend 0
Fix sscanf format spotted by cppcheck. Improve alignment. Untested.
Correct loglevel.
PR53218 Allow for longer worker names and make truncation a non-fatal error...
mod_proxy: add missing APLOGNOs.
SECURITY (CVE-2014-0117): Fix a crash in mod_proxy. In a reverse proxy configuration, a remote attacker could send a carefully crafted request which could crash a server process, resulting in denial of service. Thanks to Marek Kroemeke working with HP's Zero Day Initiative for reporting this issue. * server/util.c (ap_parse_token_list_strict): New function. * modules/proxy/proxy_util.c (find_conn_headers): Use it here. * modules/proxy/mod_proxy_http.c (ap_proxy_http_process_response): Send a 400 for a malformed Connection header. Submitted by: Edward Lu, breser, covener
mod_proxy: add ap_proxy_define_match_worker() and use it for ProxyPassMatch and ProxyMatch section to distinguish between normal workers and workers with regex substitutions in the name. Implement handling of such workers in ap_proxy_get_worker(). PR 43513
Fix C99 (or later only) comment.
mod_proxy: Don't limit the size of the connectable Unix Domain Socket paths. Since connect() to UDS path is used at several places, introduce ap_proxy_connect_uds() in proxy_util.
revert 1602523, until there is understanding about what the actual issue and "requirement" are.
Allow for "magic" scheme "auto" which makes the scheme of the backend worker match whatever the scheme of the incoming request was... For example: ProxyPass / auto://foo.example.com/ If the incoming request is http:.../lala then the resultant will be http://foo.example.com/lala If it's wws:.../lolo then we'd send wws://foo.example.com/lolo
mod_proxy: follow up to r1601291. Since deferred_write_pool is needed by the core_output_filter and is a subpool of the connection, shutdown in a pre_cleanup of the connection's pool to avoid a freed memory access (SEGV). Reported By: takashi
mod_proxy: Shutdown (eg. SSL close notify) the backend connection before closing.
Revert r1601285 and r1601283. Shouldn't have commited the latter without disussing it on dev@. Since the former creates upper APLOGNOs, revert and then recommit with the reverted next tag number.
mod_proxy: Shutdown (eg. close notify) the backend connection before closing.
Follow-up to r1592529: Define default port for "scgi" schemes (as chosen by mod_proxy_scgi) in a common location. Suggested by: jailletc36
mod_proxy_scgi: Support Unix sockets. ap_proxy_port_of_scheme(): Support default SCGI port (4000).
Clarify an existing requirement of the server_portstr parameter to ap_proxy_determine_connection(): it must be a buffer of at least one byte in size. (And don't bother with using strcpy in order to zap a string.)
reformat only (get rid of unnecessary block scope)
strncmp(r->filename, "proxy:", 6) is faster than a note. Plus, allows for checking even if not due to rewrite.
mod_proxy: Preserve original request headers even if they differ
from the ones to be forwarded to the backend. PR 45387.
mod_proxy: When ping/pong is configured for a worker, don't send or forward
"100 Continue" (interim) response to the client if it does not
expect one.
mod_proxy: Add comment and avoid ternary operator as condition (no functional change).
mod_proxy: follows up r1572630. Don't reuse a SSL backend connection with no SNI for a request requiring SNI.
Redo what was reverted in r1572627. Don't reuse a SSL backend connection whose SNI differs. PR 55782. This may happen when ProxyPreserveHost is on and the proxy-worker handles connections to different Hosts.
Revert r1572606 for the time being since backport would conflict with 2.4.x's proxy_conn_rec. The uds_path field is at the end of the struct in 2.4.x but not in trunk. Fix that first, then recommit.
Don't reuse a SSL backend connection whose SNI differs. PR 55782. This may happen when ProxyPreserveHost is on and the proxy-worker handles connections to different Hosts.
Avoid useless functions calls.
Cleanup the client to backend brigade before returning an error (if any) to avoid buckets lifetime issues (backend connection's pool destroyed before request's one). PR 50335. Suggested by rpluem.
* Do not parse URL in case of regular expression as they likely do not follow the URL syntax. PR: 56074
r->filename should maintain the proxy: prefix for PHP-FPM
handle leak. thx to Y^2
Reflow logic... pull UDS stuff out
Tuck away UDS path in request-rec, since worker isn't thread-safe. Protect from NULL refs.
Adjust url as required, following what we did to r->filename. Save some cycles when searching...
make mod_rewrite and mod_proxy UDS work together...
Correct logic... if this is a UDS socket, then bypass all this.
We were not being consistent between http and others if we added the default port or not during the canonizing phase... Baseline the http method (don't add unless the port provided isn't the default).
UDS urls need to be desockified when configuring...
* Fix logical flaw introduced in r1531340. We checked for the opposite.
Rüdiger and Yann suggestions
another suggestion by Yann... use a shorter lifetime pool
typo
from an idea by Yann Ylavic <ylavic.dev@gmail.com>
move pool creation to the single instance when we need it... but be pool aware anyway, just in case.
Only assume localhost in the special case where the person just has the lone scheme after the |
ap_proxy_strncpy should correctly handle src being NULL. Actually, apr_cpystrn() should as well...
Put the uds path in its own field, and adjust the logic to look for an empty string rather than a flag.
don't use what could have been freed...
Update LOGNO
Standardize on: unix:/path/to/socket|scheme://ignored for ProxyPass UDS.
check that we have a pool...
Just use the def global... consistency is important
Ensure that userland format of UDS is the same as how it is configured, no matter how we store and use it internally.
clean up socket definition... just the sock: path matters, and we adjust relative to DefaultRuntimeDir
syntax sugar... if the worker is associated w/ a UDS, then make sure the log reporting has a visual clue.
Reformat the UDS support inline with a new naming structure. Use a flag for speed for testing.
save a possible lookup
follow-up to r1530603: fix typo/syntax error
ping tuning via Yann Ylavic <ylavic.dev@gmail.com>
mod_proxy: Add ap_connection_reusable() for checking if a connection is reusable as of this point in processing. mod_proxy_fcgi uses the new API to determine if FCGI_CONN_CLOSE should be enabled, but that doesn't change existing behavior since the connection is currently marked for closure elsewhere in the module.
No need to lower the string before removing the path, it is just a waste of time.
Allow for a simple socket check in addition to the higher level protocol-level checks for backends... Not sure if it makes sense to do both or not... Comments?
Save some cycles...
UDS subsequent request on a connection fix
s/seperate/separate/g
Potential use of uninitialized memory. Point 3 of PR 54936
conf->mutex is not used... Also, ensure that pool use is protected
NULL terminate the right buffer
mod_proxy: Use a global mutex for handling workers.
mod_proxy: Make sure we skip empty tokens when parsing the Connection header.
core: Stop the HTTP_IN filter from attempting to write error buckets to the output filters, which is bogus in the proxy case. Create a clean mapping from APR codes to HTTP status codes, and use it where needed.
hunk 1: C89 please; hunk 2: optional functions are usually declared static.
mod_proxy, mod_proxy_http: Connection headers must be stripped on the way in and out, support an optional function to handle this.
mod_proxy: Ensure we don't attempt to amend a table we are iterating through, ensuring that all headers listed by Connection are removed.
Mod_proxy used the global pool w/o mutex. fix.
mod_proxy: Ensure network errors detected by the proxy are returned as 504 Gateway Timout as opposed to 502 Bad Gateway, in order to be compliant with RFC2616 14.9.4 Cache Revalidation and Reload Controls. [Reverted by r1884280]
* Always try to reuse the address looked up for the worker if we are allowed to reuse the address. This saves DNS lookups.
* Improve reusage of already resolved addresses to avoid unnecessary DNS lookups.
message tag for dom sock
Pull out duplicated code to proxy_util...
apr trunk-able
Follow-up to r1451905 to fix NetWare/Windows compilation.
Make AF_UNIX aware... fix Windows/Netware??
Add in rough uds support (Bugx 54101) from Blaise Tarr <blaise.tarr@gmail.com>
non-existance of byrequests is not an immediate error
Allow for searching w/i shm slots for a specific worker and balancer
better logging for re-use/use of shm
Allow for setting of sticky session split char... Bugz 53893
Log whether or not the restore from shm actually resulted in a match of shm data, or whether it was stale.
Use identifying server_rec info when we know we have unique and useful data :)
Persist local balancer-manager changes across restart/graceful.
Don't just willy nilly slurp double slashes... just handle the end-case where a '//' is not intended (due to how PPR is stored)
When concatting for PPR, avoid cases where we concat ".../" and "/..." to create "...//..."
Avoid the overhead of creating and grabbing a uuid for the balancer nonce if we're never going to use it.
* modules/proxy/proxy_util.c (ap_proxy_location_reverse_map): Fix to skip balancer:// prefix where applicable. Reviewed by: rjung, rpluem PR: 45434 (regressed since 2.2)
* modules/proxy/proxy_util.c (ap_proxy_checkproxyblock): Add error handling for apr_sockaddr_ip_getbuf(), minor code style tweaks.
* modules/proxy/proxy_util.c (ap_proxy_checkproxyblock): Fix memory leak/thread-unsafe use of pconf. Submitted by: rpluem, jorton
Fix handling of ProxyBlock if a forward proxy is configured, and fix mod_proxy_connect to avoid an inappropriate DNS lookup: * modules/proxy/proxy_util.c (ap_proxy_checkproxyblock): Take hostname argument, make address argument optional. Check names against hostname arg, omit if addr not given. (ap_proxy_determine_connection): Adjust for the above; pass hostname from URI not the next hop. * modules/proxy/mod_proxy_ftp.c (proxy_ftp_handler): Adjust for ap_proxy_checkproxyblock change. * modules/proxy/mod_proxy_connect.c (proxy_connect_handler): Adjust similarly, and avoid the DNS lookup on the request-URI hostname if a proxy is used. * include/ap_mmn.h: Bump MMN. PR: 43697
Various code clean up Submitted by: Christophe JAILLET <christophe jaillet wanadoo fr> PR: 52893
* Add the forcerecovery balancer parameter that determines if recovery for balancer workers without considering the retry value of workers is enforced. There might be cases where an already overloaded backend can get into deeper trouble if the recovery of all workers is enforced without considering the retry parameter of each worker
optimize this... Do strlen() ONLY if we need to check for overflow.
Cure size_t abuse, curse some recent code [recommit good bits from r1227852]
Revert a whole mess that was not ment to be committed
Cure size_t abuse, curse some recent code
Update the comment and checks around address/connection reusability in mod_proxy. The checks and comment are a vestige of 2.0.x mod_proxy_http where the backend proxy conn was stored in the frontend r->conn_config, but ap_proxy_determine_connection() doesn't associate the frontend and backend conns like this.
Various code cleanup to avoid compiler, cppcheck, or clang warnings:
modules/debugging/mod_firehose.c: Make some internal functions static
(to do: logs_cleanup() is unused)
modules/filters/mod_charset_lite.c: Remove dead assignments
modules/filters/mod_include.c: likewise
modules/metadata/mod_usertrack.c: likewise
modules/proxy/mod_proxy_ftp.c: likewise
modules/ssl/ssl_engine_pphrase.c: likewise
modules/proxy/mod_proxy_balancer.c: likewise;
Remove NULL check that can never happen
modules/proxy/proxy_util.c: Axe NULL-check that can never happen and if it
would, it would just mask another bug
os/unix/unixd.c: likewise
modules/http/http_filters.c: Remove sub-condition that is always true
modules/lua/mod_lua.c: Add default cases to switch statements
modules/generators/mod_autoindex.c: Unsigned value can never be < 0
server/util_expr_eval.c: Fix compiler warnings with VC and on OS2
mod_proxy: Remove dead code ap_proxy_table_unmerge().
mod_proxy: Remove ap_proxy_make_fake_req() from the public API.
mod_proxy: Remove dead code.
mod_proxy: Make sure we populate the client IP from the peer IP for proxy fake requests.
mod_proxy: Remove ap_proxy_date_canon() from the public API.
mod_proxy: Remove ap_proxy_buckets_lifetime_transform() from the public API.
mod_proxy: Remove ap_proxy_liststr(), it duplicates ap_find_token().
mod_proxy: Move ap_proxy_string_read() out of the public API into mod_proxy_ftp.
Add lots of unique tags to error log messages
mod_proxy: Make ap_proxy_retry_worker() into an optional function. Allows mod_lbmethod_bybusyness, mod_lbmethod_byrequests, mod_lbmethod_bytraffic and mod_lbmethod_heartbeat to be loaded without mod_proxy yet being present, which happens when modules are loaded in alphabetical order.
Use 2 sep hashing functions to account for collisions... Safe enough
Ensure casing doesn't matter... (we know balancer->s->name is lc)
Some optimization... we have a hash, use it. Quicker than all these string comparisons.
Logic which makes the balancer "active" or not... so we can turn them off and on as needed.
Pull this out as a sep function to allow for full vhosting adjustment between get and define balancers...
"Final" (I hope) proxy struct/API changes in order to support dynamic balancer allocation AND dynamic vhost-balancers
Error-check.
Move useful and shared balancer stuff to the shm slot... Provide hooks for post-config/runtime addition of balancers.
Allow compile time changes of field sizes and be more "clear" regarding the diff between balancer and worker slots...
Never used.
Log fatal errors at level CRIT, not at some random level from DEBUG to EMERG
great proxy logging cleanup:
* remove "proxy:", "FCGI", etc. prefixes and pid which are now
included in the error log format
* propagate frontend request's logconfig to backend request
* use ap_log_rerror where possible
* remove obsolete APLOG_NOERRNO
Modify placement of PROXY_DECLARE_DATA to fix build problems with MSVC, reported by Gregg L. Smith
There is absolutely no reason to have two 4k-sized constant strmatch patterns in each per-dir config.
Remove ap_proxy_removestr(). It has been unused for a decade and has quadratic memory usage.
More cleanup: Expand tabs and some more indentation fixes No functional change
Save one call of ap_escape_html on the uri. Reflow code to be more readable. Submitted by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> PR: 51697
Add wrappers for malloc, calloc, realloc that check for out of memory situations. Use them in most places where malloc, and friends are used. This results in clean error messages in an out of memory situation instead of segfaulting or silently malfunctioning. In some places, it just allows to remove some logging code. PR 51568, PR 51569, PR 51571.
mod_proxy: enable absolute URLs to be rewritten with ProxyPassReverse, e.g. to reverse proxy "Location: https://other-internal-server/login" with ProxyPassReverse https://public-address/ https://other-internal-server/
Various code cleanup PR: 51398 Submitted by: Christophe Jaillet <christophe jaillet wanadoo fr>
Avoid some memory allocations by using apr_table_setn where the string arguments are const. Submitted by: Christophe JAILLET <christophe jaillet wanadoo fr> PR: 51357
Replace some "apr_p[c]alloc / memcpy" constructions into a single apr_pmemdup()s PR: 51214 Submitted by: Christophe Jaillet <christophe jaillet wanadoo fr>
If we change the LB method, or we take add in a new
balancermember ("enable" her), then we need to have the
LBmethod reset its params as required...
Resolve disagreement with its prototype
No longer depend on how fork() works when laying out segments... function pointers in shm are... nasty ;)
move function...
Ugg... unbalanced workers *do* need a mutex as well (good catch Rüdiger!)
Remove the thread mutex from the worker... it really should be in the balancer. Thus we have global and thread for the balancer. Use global when updating the full, shm list of workers; use thread when being local.
Here we go... we can now, via balancer-manager, add new workers to existing balancers. Still work to be done, like error checking that we aren't trying to add more than we can (right now, it fails, but it would be nice to handle it nicer), disabling and *deleting* workers we don't want anymore, the actual drain method, etc... but this is some major goodness.
No need to pollute the whole namespace...
Optimize... create a struct to hold the worker status info (need to align w/ set_params later on) and note when we update timestamps
Again, abstract out the worker status magic to it's handled by funcs...
Move the setting of flags into a func... that way there is one place to maintain as the number of flags grow... Since we are using just the bits, make unsigned.
Move the lbmethod to also allow it to be adjusted runtime
Move some "nice to be able to change" balancer stuff to shm
Move nonce to balancer struct and add logic for nonce and growth. Adjust mmn and update doccos
Set some defaults... do it here @ worker creation, not initialization
whitespace
Builds and passes tests as prefork... good save state
Adjust for conf->workers being *proxy_worker and balancer->workers being **proxy_worker
More movement to shared stuff...
OK... a good commit point (we don't quite compile yet though...) Pull out the worker scoreboard cruft and start moving most worker stuff to shm. Use slotmem for workers and provide space for growth. Redo logic: ap_proxy_define_* ap_proxy_create_* ap_proxy_initialize_* Right now just for workers, but lay framework for balancers as well. The idea is to break out the functional parts to make it easy for dynamics. Defining is simply describing the worker, and tucking that info away. When we create, we go ahead and create the shared memory, etc... Initialize is simply to allow child process to access the shm...
Use branch now
* Be ANSI C compatible
mod_proxy_http: Allocate the fake backend request from a child pool of the backend connection, instead of misusing the pool of the frontend request. Fixes a thread safety issue where buckets set aside in the backend connection leak into other threads, and then disappear when the frontend request is cleaned up, in turn causing corrupted buckets to make other threads spin.
Fix a spin should we encounter a corrupt bucket from the proxy.
* The concept of the cleaned flag is flawed: Once we returned the connection to the pool we cannot longer rely on it as another thread could have leased the connection in the meantime and might have modified it. BUT: We only use this flag once we returned the connection to the pool. So signal that we returned the connection to the pool by something that is local to the thread, in this case set backend to NULL if we already have returende the connection.
Fix 'comparison is always false due to limited range of data type'. One-bit flags should really be unsigned
Revert r1035504, this was the wrong way to do it.
Fix pool lifetime issues when the proxy backend connection is terminated early by forcing a setaside on transient buckets placed in the brigade by mod_ssl. This has the effect of extending the lifetime of buckets until the end of the request. This is a variation on the original fix for this problem, which added transient buckets to be setaside later in the process.
Fix mod_proxy to support setting source address. PR 29404 Seems lots of users want this, and a couple of them have recenly asked nicely
log how long worker will be disabled
Followup to r1026665: reset "cleaned" flag in ap_proxy_acquire_connection().
Fix a number of typos and misspellings. Reported by Ville Skytta. PR48496.
mod_proxy: Release the backend connection as soon as EOS is detected, so the backend isn't forced to wait for the client to eventually acknowledge the data.
Optimise the set flags in the mod_proxy structures.
Fold in possible use of FNV if desired
ap_proxy_retry_worker() only needs to spit out one debug message, and it should indicate whether or not retry will be attempted
Prevent race condition See http://www.mail-archive.com/dev@httpd.apache.org/msg47171.html
* The forward and reverse worker do not keep connections alive, so ensure that mod_proxy_http does the correct thing regarding the Connection header in the request.
Fold in some stuff to allow for matching of workers to their actual scoreboard slots, independent of slot ID.
- Remove a load of unused variables (or variables that are set but never read). - Move some declarations into the correct #ifdef scope. I couldn't compile/test netware, but the changes look obvious enough.
Allow for modules to keep track of worker slot numbers themselves if they want, by allowing for worker create/alloc functions to take a slot number id. Done via _wid() variants of 3 proxy funcs.
- Be less verbose at levels INFO and DEBUG in mod_proxy* and mod_ssl - Add some trace logging to core and http
Use the new APLOG_USE_MODULE/AP_DECLARE_MODULE macros everywhere to take advantage of per-module loglevels
log issues with CONNECT at request scope instead of server scope
Handle the pathological case of a connect backend that sends us an unexpectedly huge response.
We need to copy the forward proxy data, because it might be reused during later requests using the same backend connection.
Limit sscanf format to the number of chars actually needed and buffer size provided to prevent buffer overflow.
Support remote https proxies by using HTTP CONNECT. PR: 19188 Submitted by: Philippe Dutrueux <lilas evidian.com> Reviewed by: rjung
fix maintainer mode ap_strstr() argument mismatch warnings
Fix r893871 as noted by rpluem, to remove risk of returning a relative-ised URL, and fix erroneous CHANGES bug attribution.
Proxy: fix ProxyPassReverse with relative URL PR 38864
Don't set worker->s if already set (for balancer using slotmem for examples).
Return APR_EGENERAL for the error
Allow the load balancing method to "create" workers.
* Fix syntax error
optimize.... rp! :)
Change the order of mod_proxy and mod_proxy_balancer child_init(). Change the balancer workers area to the address of workers instead copying the workers. Arrange lbmethod accordingly. Move the creation of conf->forward worker to mod_proxy child_init().
Explain empty code block (empty for now, that is) :)
* Silence compiler warning.
As mentioned inline in comments, correctly handle more sophisticated
transformations which currently fail for balancer://foo targets, but
work just fine with other ProxyReverse targets.
The balancer comparison is a bit trickier. Given the context
BalancerMember balancer://alias http://example.com/foo
ProxyPassReverse /bash balancer://alias/bar
translate url http://example.com/foo/bar/that to /bash/that
E.g. there may be several different url-suffixes (1st order) of any
particular BalancerMember set e.g. /app1, /app1 and /appbeta while
there may be additional suffixes associated with the actual
ProxyPassReverse directive. Neither were properly reversed, now
both should be properly handled.
One *critical* assumption;
BalancerMember balancer://alias/foo http://example.com/bar
should be documented as a meaningless construct, since one cannot
have two members, balancer://alias/foo and balancer://alias/bar,
and the balancer member structures discard this path.
Note one more existing error case as an XXX comment due to invalid
uri comparisons.
REVERT 771581 modules/proxy/ changes (miscommitted)
complete r771579, refactoring mod_watchdog as loadable on win32
Fold in initial template for methods to be able to reset (initialize) and "age" their data, useful when adding new workers, or when workers come back into the fold.... Logic and code to come in a bit :)
remove TPF support
remove BeOS OS support
Note for self: PPR drops the path info so we don't really honor PPR /foo balancer://bar/jimbob in a way that we would expect.
* Rip out the old flushing approach for solving lifetime issues between the backend connection bucket allocator and front end connection bucket allocator. Instead copy the buckets from the backend over to ones that have been created using the front end bucket allocator. For metabucket this is done by recreating them, for data buckets this is done by reading them and putting the read data in a transient bucket. PR: 45792
* Improve the way to detect whether buckets in the filter chain need to be flushed by using the main requests bytes_count field instead of the subrequest field. * Do not reset conn->need_flush. This prevents SegFaults from not flushing buckets in the filter chain. PR: 45792
* Revert r703998. The call to apr_socket_timeout_set before apr_socket_connect already sets the socket to non-blocking mode because the timeout of the socket is -1 after creation. A further call to apr_socket_timeout_set (after the connect call does not do this, because the old and the new timeout are >=0). The further code expects the socket to be in non-blocking mode, otherwise we have regressions with ssl. This can be notified by running t/ssl/proxy on 2.2.x which runs much much slower with the patch applied. This does not happen on trunk because the socket is set back to non blocking by the core output filter (async write completion).
* Make the connection timeout to backends work by temporarily setting the socket to non blocking mode. Submitted by: Matt Stevenson <mavricknzwork yahoo.com> Reviewed by: rpluem
Allow for the smax param to be 0, so that all connections are available to be dropped, should they be idle long enough. PR 43371
* Introduce environment variable proxy-initial-not-pooled to avoid reusing pooled connections if the client connection is an initial connection. This avoids the "proxy: error reading status line from remote server" error caused by the race condition that the backend server closed the connection after the connection check on our side and before our data reached the backend. Yes, this downgrades performance, especially with HTTP/1.0 clients. Hence it is configurable and off by default. PR: 37770
* Add the possibility to set a separate connection timeout for backend workers. PR: 45445 Submitted by: rahul <rahul sun.com> Reviewed by: rpluem
* Remove call to apr_reslist_destroy. With the latest trunk version of apr-util this causes a lock during shutdown as at the point of time we would execute apr_reslist_destroy the reslist is already destroyed, because we are in a cleanup of the same pool where the reslist registered itself as precleanup. With apr-util 1.3.x calling apr_reslist_destroy is not really useful and needed in this case as we are in a cleanup that was registered against the same pool that is used by the reslist. As it was registered *after* the reslist was created it just runs *before* the reslist cleanup would run. This is somewhat pointless here and we could leave the job of destroying the reslist to the reslist cleanup.
Use what we have. This logic belongs here anyway...
Note why we aren't bothering to check that we have a valid octet here.
Add in mod_jk's disablereuse analog for mod_proxy.
* Fix a typo (timout -> timeout) PR: 44360 Submitted by: Rainer Jung <rainer.jung kippdata.de> Reviewed by: rpluem
Now let things like ProxyPassReverse /foo balancer://bar work "as expected" :) :)
* Using the reslist pool for the proxy_conn_rec structure introduces a memory leak when connections get created and destroyed frequently by the reslist (e.g. destroying idle elements of the reslist). So use the subpool dedicated for the proxy_conn_rec structure to allocate the memory for the structure itself.
* Only sent a flush bucket down the chain if buckets where sent down the chain before that could still be buffered in the network filter. This is the case if we have sent an EOS bucket or if we actually sent buckets with data down the chain. In all other cases we either have not sent any buckets at all down the chain or we only sent meta buckets that are not EOS buckets down the chain. The only meta bucket that remains in this case is the flush bucket which would have removed all possibly buffered buckets in the network filter. If we sent a flush bucket in the case where not ANY buckets were sent down the chain, we break error handling which happens AFTER us.
* Tag the pools appropriately to ease memory debugging.
* Fix a SEGFAULT by ensuring that buckets that may have been buffered in the network filters get flushed to the network. This is needed since these buckets have been created with the bucket allocator of the backend connection. This allocator either gets destroyed if conn->close is set or the worker address is not reusable which causes the connection to the backend to be closed or it will be used again by another frontend connection that wants to recycle the backend connection. In this case we could run into nasty race conditions (e.g. if the next user of the backend connection destroys the allocator before we sent the buckets to the network). Remark 1: Doing a setaside does not help here as the buckets remain created by the wrong allocator in this case. Remark 2: Yes, this creates a possible performance penalty in the case of pipelined requests as we may send only a small amount of data over the wire.
* Do not register connection_cleanup as cleanup for the conn->pool. In the past it was needed to register connection_cleanup as a cleanup for the frontend connection memory pool (c->pool) to ensure that connection returns into the connection pool if the memory pool of the frontend connection memory pool gets destroyed / cleared. Now we ensure explicitly the connection returns to the connection pool once we finished handling the request.
* Fix another memory leak related to PR 44026. Now that we keep the connection data structure alive in the reslist, the live time of c->pool is too long. r->pool has the correct live time since rp dies before r.
* Use a separate subpool to manage the data for the socket and the connection member of the proxy_conn_rec struct as we destroy this data more frequently than other data in the proxy_conn_rec struct like hostname and addr (at least in the case where we have keepalive connections that timed out and were closed by the backend). This fixes a memory leak with short lived and broken connections. PR: 44026
* Enable the proxy to keep connections persistent in the HTTPS case. Basicly the persistence is created by keeping the conn_rec structure created for our backend connection (whether http or https) in the connection pool. This required to adjust scoreboard.c in a way that its functions can properly deal with a NULL scoreboard handle by ignoring the call or returning an error code.
Semi-revert 583466; just remove the ap_proxy_isvalidenc() function until we canonically set the "allowed" list.
Abstract out "verification of valid encoding" via ap_proxy_isvalidenc(). Now we can use it in other proxy protocols.
Fix keepalive to proxy backend PR 43472
lb_score is bigger proxy_worker_stat. Prevent wasting space.
* This check is now part of the PROXY_WORKER_IS_INITIALIZED macro.
Maintain the illusion. It's not worth my time or energy to care about this anymore.
Escape error-notes correctly PR 40952
Check all IP addresses listed in ProxyBlock PR 36987
mod_proxy: Don't lose bytes when a response line arrives in small chunks. PR 40894
Reinstate the lb_score patch, but, protect against conflicting leakage as per jfc proposal.
*) mod_proxy: Improve network performance by setting APR_TCP_NODELAY
(disable Nagle algorithm) on sockets if implemented.
PR 42871 [Christian BOITEL <christian_boitel yahoo.fr>, Jim Jagielski]
Improvement suggested by rpluem to proxy_date fix
Fix buffer overflow in date handling PR 41144 (Davi Arnaut)
Fix the timeout logic. The order is now: 1 - worker->timeout 2 - proxy_conf->timeout 3 - server->timeout. ap_get_module_config() is not perfect by that is easy to port back to 2.2.x.
Arrange the proxy timeout behaviour.
Abstract out (kinda) MSG_PEEK. This way when we actually update APR and apr_socket_recvfrom() to know about APR_MSG_OOB, APR_MSG_PEEK, etc... this code won't need to be adjusted. Also, make nicer for 2.2 backport.
Isolate the unlock return vals
Failure to unlock is very nasty, so log it to help with troubleshooting.
* Ensure that at least scheme://hostname[:port] matches between worker and URL when trying to find the worker that fits best to the given URL. PR: 40910
Add alternate is_socket_connected by using APR functions.
Remove alternate is_socket_connected detection.
* Assign rv in the if statement to return the correct error code and more important an error code at all, as currently APR_SUCCESS is returned in the error case. PR: 40865 Submitted by: Andrew Rucker Jones <arjones simultan.dyndns.org> Reviewed by: rpluem
Add missing include for experimental is_connected.
Cleanup alternate is_connected method. It works for sure on win32,linux and solaris. I do not have access to other platforms so can not verify it works on them.
Add alternative is_socket_connected implementation. It works on win32 and linux for sure, so that's why I put it inside the #ifdef. Of course something like that needs APR abstraction.
Enable retry=0 for the worker. This allows to have an option to always retry the workers in error state instead using a specified time.
* PROXY_WORKER_IS_INITIALIZED checks if PROXY_WORKER_INITIALIZED is set in worker->s->status, but we need to check here if it is set in worker->status. As PROXY_WORKER_INITIALIZED is set in worker->s->status in ap_proxy_initialize_worker_share, which is called just before ap_proxy_initialize_worker we never initialize the worker. This is very bad as we create no reslist for the pool in this case.
PR#37770: Don't try to use dead backend connection in proxy (Patch from Olivier BOEL)
Fix tabs introduced in r429875
PR#38448: don't URLencode tilde in path component On reflection, this patch makes sense (since the URL spec got updated), and it's trivial.
Fix style (indentation at least) to apache style rules before reviewing top of this. No functional change.
Cleanup: Remove close_on_recycle from proxy_conn_rec. It behaves the same as close.
Cleanup: Remove checking for pre 2.0.46 MMN. It is obvious that the new mod_proxy will never be backported to the 2.0 branch.
Support environment variable interpolation in reverse proxy configuration
It never fails. I sit on a patch for awhile and it's not until almost right after I commit it that I think "hey, there's a better way to do that." Anyway, I was never happy about the code duplication of the primary/standby checks... This fixes that.
Add in hot-standby balancer member. If all other members are disabled or not-usable, ONLY THEN will the hot standby's be used.
update license header text
Clean up some proxy macros. Avoid the use of magic numbers, and instead use pre-defined defines. Also, ensure that usable workers have been initialized :) Allocate a bit for hot standbys. Adjust so that normal "usable" workers don't count these.
* modules/proxy/proxy_util.c (ap_proxy_initialize_worker): Fix gcc strict-aliasing warning.
Update the copyright year in all .c, .h and .xml files
Fix for platforms without threads: inreslist exists only if APR_HAS_THREADS is set
PR#39321 - don't segfault if a bad URL is specified in ProxyPass
* Avoid calling ap_proxy_http_cleanup twice as this releases a connection
from the connection pool twice. This causes this connection to be present
in the connection pool twice. Thus it may be used by different threads
at the same time which causes many troubles (segfaults in this case).
Furthermore implement a logic to prevent double releases to the connection
pool if they are triggered by buggy code and log an error message in this
case.
- mod_proxy_http.c: remove double calls to ap_proxy_http_cleanup
- proxy_util.c: Add logic to prevent double releases of a
connection to the connection pool.
PR: 38793
* Check for an existing socket independant of an existing connection record. This ensures that requests handled by the '*' worker are sent to the correct backend server. PR: 39253
Document the removal of the flushing bandaid to a runtime param. Since other protocols might benefit from this, remove the ajp_ prefixes, to make it more generic looking.
Make AJP flushing admin configurable.
*) mod_proxy: Fix KeepAlives not being allowed and set to
backend servers. PR38602. [Ruediger Pluem, Jim Jagielski]
Also, document previous patch:
*) Correctly initialize mod_proxy workers, which use a
combination of local and shared datasets. Adjust logging
to better trace usage. PR38403. [Jim Jagielski]
OK, handle better the initializing of worker, separating "shared" and "local" inits, and being aware that if the shared aspects are initialized, this doesn't mean the local ones are :)
I wondered why I never saw the scoreboard init entry; now I know. The logic flow was slightly askew.
* If a subrequest has a broken backend also set no_cache for the main request
and ensure that the chunk filter does not sent the last chunk marker in this
case.
modules/http/chunk_filter.c: Memorize HTTP_BAD_GATEWAY error buckets that
had been seen in filter context to ensure
that we do not sent the last chunk marker in
this case.
modules/proxy/proxy_util.c : Set no_cache also for main request if we are
a subrequest.
Thanks to Joe Orton and André Malo for the "invented unique pointer" trick.
* Correctly signal broken backend connections up the chain also for the ajp backend (see also r357461). Furthermore move common code in mod_proxy_http.c and mod_proxy_ajp.c into a new function (ap_proxy_backend_broke) in proxy_util.c. modules/proxy/mod_proxy_ajp.c : Signal broken backend connection for ajp backend modules/proxy/proxy_util.c : Add ap_proxy_backend_broke modules/proxy/mod_proxy_http.c: - Use ap_proxy_backend_broke - Return DONE also if backend broke modules/proxy/mod_proxy.h : Add declaration of ap_proxy_backend_broke
OK, while we ponder how best to reuse conn pools for workers, clean this section up.
Until we determine whether we should dip into the connection pool each time, rearrange the logic to avoid a double check which is unneeded. No matter what (if it's reusable or not) a null conn->hostname needs to be set. At that point it doesn't matter if we're not reusing it or if the string comparison happens, since what we would do in that case has already been done.
Check that conn->hostname is valid before we do the string comparison
Fix case where a shared keepalive connection results in different workers from being accessed. Try to be wise and fast about the tests before we do the expensive string comparison.
No functional Change: Removing trailing whitespace. This also means that "blank" lines consisting of just spaces or tabs are now really blank lines
Move to a different impl which was my 1st concept and that which Ruediger likes. Instead of adjusting pointers to areas within the URL, simply copy it over and change it as needed. Easier logic and not that much slower for normal cases.
Fix a problem where we are doing a case insensitive match between the worker and the URL. Instead, only the scheme and hostname are insensitive, the rest should be case sensitive.
* Fix PR36906 by not lower caseing the whole worker name. Only lower case the schema when storing the worker url. Thus preventing case sensitive URI's in BalancerMembers to get broken.
Performance Tune: Do the cheap and fast length check before we bother doing a char-by-char comparison.
Fix -Wall warning... Thanks to Joe for the head's up!
mod_proxy_balancer: BalancerManager and proxies correctly handle member workers with paths. PR36816. [Ruediger Pluem, Jim Jagielski]
* Fix style nits (whitespaces / formats of comments). No Functional changes.
de-ref bug - balancer is always not null since it's &balancer from the caller.
Proxy balancer lbmethods are now registered as providers and not via hooks. Move various find() functions back to mod_proxy_balancer where they belong :)
Restruct of lbmethod find-best algo's, bypassing hook mechanism. Looking into provider method, but until then... Prevent core dump when balancer not in vhosts
Get rid of some warnings
simple, functional interface to add additional balancer lb selection methods without requiring code changes to mod_proxy/mod_proxy_balancer; these can be implemented via sub-modules now. Let the games begin...
Fix mess left by incompleteness of previous ProxyPassReverse fix
Add status param option so that current worker status can be changed via editing httpd.conf instead just using web page. Seems the ApacheCon is paying of. Thanks to Sander for pointing that out.
- Use apr_uri_t to parse the proxy URL, instead of munging it by hand. Original code did not handle IPv6 Addresses correctly.
Revert the latest hack for setting query string. The balancer needs canon_handler.
Add r->args to uri->query if the original r->uri did not contain query string. This fixes #34266.
Remove formatting characters from ap_log_error() calls. These were escaped as fallout from CAN-2003-0020. Submitted by: Eric Covener <ecovener gmail.com> Reviewed by: Jeff Trawick
* modules/proxy/proxy_util.c (ap_proxy_pre_request): Log an error when all workers are busy.
Update copyright year to 2005 and standardize on current copyright owner line.
Close PR 32459, 15207. API change for PROXY_DECLARE ap_proxy_canonenc()
Add in a weighted byte count of all traffic (in and out) as an alternative balancing method. We do not "adjust" the byte count wrt scheme or method, simply by factoring in the lbfactor value.
Rewrite Cookies and other headers in mod_proxy_ajp as configured with
ProxyPassReverse and ProxyPassReverseCookiePath.
* modules/proxy/proxy_ajp.c
(ap_proxy_ajp_request): Update call to ajp_parse_header().
* modules/proxy/ajp.h
(ajp_msg_get_string): constify argument.
(ajp_parse_header): Add conf argument.
* modules/proxy/proxy_util.c
(ap_proxy_location_reverse_map): Promoted from mod_proxy_http.
(ap_proxy_cookie_reverse_map): Promoted from mod_proxy_http;
renamed from proxy_cookie_reverse_map.
* modules/proxy/ajp_msg.c
(ajp_msg_get_string): constify argument.
* modules/proxy/proxy_http.c
(ap_proxy_location_reverse_map, proxy_cookie_reverse_map: Removed.
(process_proxy_header): Update header mapping table to reference the now
public functions.
* modules/proxy/ajp_header.c
(ajp_unmarshal_response): And conf argument. Reverse map cookies and
other headers.
(ajp_parse_header): Update call to ajp_unmarshal_response().
* modules/proxy/mod_proxy.h
(ap_proxy_location_reverse_map, ap_proxy_cookie_reverse_map): Declare
functions that have been promoted to be public.
(ap_proxy_header_reverse_map_fn): Typedef for the signature of the above
two functions (currently used in mod_proxy_http).
Kill the pool cleanup when closing non cachable connection.
Fix closing connection for non-cachable proxies. Do not close the entire connection, just socket if opened.
Close the connection for generic forward and reverse proxy workers.
Simplify connection address and port resolving for non-cachable workers (gereric forward and reverse proxy).
Do not cache hostname and port for forward and reverse generic proxy workers.
Make proxy address cache thread safe and available only to pooled workers.
mod_proxy: Respect errors reported by pre_connection hooks.
wrong format string was converted to APR_PID_T_FMT
Add generic reverse proxy worker so that module-driven reverse proxying works. It is enabled by default, but IMO it should be configurable like any other worker so we can limit the number of outgoing connections.
Use APR_PID_T_FMT instead %d for pid logging.
Make sure that worker initialization is done only once per worker, as well the initialization of shared memory slot. Also log the child pid so that logging makes sense.
general property cleanup
Do not use reslist if it wasn't created. This fixes cases on prefork with threads enabled and hmax set.
Add server_rec to initialize_worker_share so that it get's correctly.
Reformat code to no tab and no CR-LF
Make sure that we don't increase balancer worker count on gracefull restart. Thanks to Christian...again.
Remove proxy_runtime_worker and make the things simpler by sharing runtime status accross child processes. Also make sure that shared data is initialized after the scoreboard is created.
Make worker retry algorithm much simpler removing incremental timeout increase.
Fix worker retries and limit default retry timeout to 10 minutes. Thanks to Christian von Roques for spotting that.
Register output function so that core can retrieve it and set scoreboard accordingly
Fix typo -- uri is constructed when there is no proxy in chain.
No need to allocate hostname on each request for persistent connections.
Always use prive connection pool for allocation, and call apr_pool_clear when the connection is closed on recycle, so that we don't leak on persistent connections.
Use worker as a parameter for connection_constructor. When min connections is greater then zero connstructor is called when the reslist is created.
Do not add forward proxy to other proxies. Use new function to create the worker that isn't by default added to the list of other workers. This enalbles mixing forward and reverse proxy functionality on the same box.
Remove the unused close_connection function.
Make sure that the connection pool cleanup is called before the reslist is destroyed.
Just like for balancers initialize the workers on post_config instead on first request to skip the race condition.
Remove the proxy_conn_cleanup. The socket will be closed when the pool is cleared or destroyed by it's own cleanup routine. Also skip clearing sockets if the workers pool is destroyed, cause the pool might already be destroyed.
Move the shared scoreboard data initization to the balancer module. The initialization is done like for the worker, on first request. This is done because during config phase we don't have a scoreboard to allocate from. Thanks to Chritian von Roques for spotting that.
Remove unneeded variable. No functional change.
Make sure that the status is not overwritten.
spellcheck
First try the reverse proxies. If there is no worker defined for requested uri use the forward worker, but only for PROXYREQ_PROXY requests.
Add PROXY_WORKER_IGNORE_ERRORS flags that disables puting the entire worker in retry mode, cause for forward workers the remote is not fixed.
Add sockaddr to proxy_conn_rec. Using the workers address is unsafe for forward proxy workers, cause unlike reverse proxies the forward can have each connection with different remote address.
Use forward worker for all PROXYREQ_PROXY requests.
Fix some compiler warnings in proxy PR: Obtained from: Submitted by: Geoffrey Young <geoff@modperlcookbook.org> Reviewed by:
axe some unused variables and don't log an error code that hasn't been initialized
Fix setting of hostname for backend. The now reflects the real name withouth scheme:// prefix. Submitted by: mturk
Added dynamic worker limit. It ensures that the addng dynamic workers to the balancers never excedes that limit. Submitted by: mturk
Add some extra debugging to ease the bug chasing :) Submitted by: mturk
Implement the worker retry functionality. It uses either worker->retry option or default 60 second retry that is on each revolution extended by another 60 seconds. Submitted by: mturk
Shared memory support. The shared memory is inside apache scoreboard. If it ever gets back ported we could use it on 2.0 Submitted by: mturk
Add route and redirect to worker so they can be set trough configuration. Add worker status flag definitions. Submitted by: mturk
Set initial lbstatus to calculated lbfactor. Submitted by: mturk
Fix the assertion bug for reslist caused by returning the same connection back to the pool twice. Submitted by: mturk
Change the scheme handler hook to include worker. We will probably need the balancer too, but that can wait for now. Other undocumented changes (shame,shame) in proxy_util.c Submitted by: mturk
Added close_on_recycle flags for creatin connections. This flag enables to distinguish between connection types. Also added a pool cleanup bound to connection pool that recycles the connection when client disconnects from server. Submitted by: mturk
Added acquire and release connection functions. The acquire pulls the connection from the pool. The release returns the connection to the pool. Submitted by: mturk
Added ap_proxy_connection_create function that makes conn_rec for opened connection. It is used bt http(s) and ftp for bounding the backend connection to client connection with the same id's. Submitted by: mturk
Add ap_proxy_connect_backend that will replace ap_proxy_connect_to_backend. It uses worker params for setting socket options and checks for conection's connect status. It works both on empty and acquired connections. Submitted by: mturk
Change the way the prefork connection is created. Use the same constructor as for theaded mpm's. Added API's for destroying and closing connections Submitted by: mturk
Added acquire timeout for obtaining resources from reslist. The timeout is in milliseconds to enable quick return in case the reslist is exceded the maximum number of connections. Submitted by: mturk
Remove proxy_mdule_conf. We will change the scheme hook api to include the worker. Submitted by: mturk
Remove proxy_conn struct and add it's params to proxy_conn_rec. Use named structure type declarations. Submitted by: mturk
Use the worker pool for creating proxy_conn_poll to suppress it's destruction when cleaning worker. Submitted by: mturk
Arrange the handling of apr_array_header_t arrays. Submitted by: jfclere
Make proxy_conn always accesible for preforked MPM and APR_HAS_THREADS enabled. Add init_conn_worker for creating apr_reslist that maintains connection pool. For prefork mpm's use the single connection container. Submitted by: mturk
Add some source comments about worker DNS lookup. Submitted by: mturk
Added ap_proxy_determine_connection. It will be used both for http and ajp handlers. Started writing apr_reslist connection pool. Submitted by: mturk
Add ap_proxy_pre_request API function. It calls the actual module pre_request hook if present Submitted by: mturk
Make sure that APR_MAJOR_VERSION is defined. Submitted by: jfclere
Remove all worker/balancer API's to proxy_util. Submitted by: mturk
Missed a 2.0-HEAD proxy back-compat patch
Fix for multiple proxy bugs - review please: #10722 Reverse proxying cookies #15207 Proxy passing canonicalised URIs to backend #16812 Case-insensitivity of proxypassreverse #19317 Canonicalised URI causing infinite loop #20372 AllowEncodedSlashes May also fix 13577 (untested) This is really two fixes: 10722 and 15207; the others are trivial consequences. To make review easier, the simpler fix (#15207) is entirely contained in #ifdef FIX_15207 (new code) and #ifndef FIX_15207 (removed code)
Fix handling of IPv6 numeric strings in mod_proxy. (Some such operations would work if port was specified in url due to way parsing was performed, finding last ':' and considering everything after as the port.)
fix name of The Apache Software Foundation
fix copyright dates according to the first check in
apply Apache License, Version 2.0
update license to 2004.
switch to APR 1.0 API (which is still in flux) because of the changes to the argument lists of apr_mmap_dup and apr_socket_create, 2.1-dev won't build with apr and apr-util's 0.9 branch anymore
Fixed a segfault when multiple ProxyBlock directives were used. PR: 19023 Obtained from: Submitted by: Sami Tikka <sami.tikka@f-secure.com> Reviewed by:
finished that boring job: update license to 2003. Happy New Year! ;-))
Haven't heard any vetoes regarding this, so might as well fold it in. Not too happy about cluttering up the list of directives either. But, at least, it allows 2.0 to proxy stupid IIS as much as 1.3. Docs to come soon. PR: Obtained from: Submitted by: Reviewed by:
make the output XHTML Submitted by : Erik Abele <erik@codefaktor.de>
http occurances to update for apr_socket_opt_get/set
Timeout/time fixes
Continue the Bill Rowe apr_size_t crusade.
No crutches, people!
Cleanup proxy timeouts a bit to remove emits and make consistent to vhost timeout.
Solve the 80/20 by initializing and storing server_rec->timeout and server_rec->keep_alive_timeout in apr_time_interval_t format (in apr units, whatever they be), as both values exist to pass into APR, and all APR timeouts are in apr_time_t. Reviewed by: Cliff Woolley
stop using APLOG_NOERRNO in calls to ap_log_[pr]error()
get mod_proxy to build on EBCDIC machines
Quick fix to an incorrect comment. PR: Obtained from: Submitted by: Reviewed by:
PR: Obtained from: Submitted by: Reviewed by: Ryan Bloom remove call to ap_proxy_reset_output_filters() and the function itself. this call breaks ssl proxy when the client is using keepalives. this function is also no longer needed with recent-ish filter changes.
fix some abuse of apr_port_t (flagged by Sun WorkShop)
Change the header merging behaviour in proxy, as some headers (like Set-Cookie) cannot be unmerged due to stray commas in dates. PR: Obtained from: Submitted by: Reviewed by:
Update our copyright for this year.
implement a common function for getting a socket and trying to connect
to the target host; use that common function for proxy HTTP and proxy
CONNECT
In that new function, fix this problem:
Proxy HTTP and CONNECT: Keep trying other addresses from the DNS
when we can't get a socket in the specified address family. We may
have gotten back an IPv6 address first and yet our system is not
configured to allow IPv6 sockets.
An example host is www.ipv6.org. The first address I get back is
an IPv6 address, but my machine may not be configured to allow an
AF_INET6 socket.
Before the fix: The apr_socket() failure was fatal.
After the fix: When apr_socket() fails, we go to the next address from
the resolver, successfully create a socket in the
specified family (AF_INET), and all is well.
(an unrelated fix in this commit was to pass a server_rec in a broken
ap_log_error() call)
We need to setup protocol filters for the fake request. If we don't, then when any request filters are added, the connection filters will be lost. This fixes the proxy.
fix the linkage of and add a prototype for ap_proxy_table_unmerge()
Some browsers ignore cookies that have been merged into a single Set-Cookie header. Set-Cookie and Set-Cookie2 headers are now unmerged in the http proxy before being sent to the client. Fix a problem with proxy where each entry of a duplicated header such as Set-Cookie would overwrite and obliterate the previous value of the header, resulting in multiple header values (like cookies) going missing. PR: Obtained from: Submitted by: Reviewed by:
Cleanup some fallout from the recent install_transport_filters commit. Deprecate ap_proxy_pre_http_connection.
Change ap_get_brigade prototype to remove *readbytes in favor of readbytes. If you need the length, you should be using apr_brigade_length. This is much more consistent. Of all the places that call ap_get_brigade, only one (ap_http_filter) needs the length. This makes it now possible to pass constants down without assigning them to a temporary variable first. Also: - Change proxy_ftp to use EXHAUSTIVE mode (didn't catch its -1 before) - Fix buglet in mod_ssl that would cause it to return too much data in some circumstances
Input filtering prototype change: Socket blocking type should be separate from the input filter mode type. We also no longer look at readbytes to determine the method of filter operation. This makes the use of filters more obvious and allows a wider range of options for input filters modes. To start with, the new input filter modes are: AP_MODE_READBYTES (no more than *readbytes returned) AP_MODE_GETLINE (old *readbytes == 0 case) AP_MODE_EATCRLF (old AP_MODE_PEEK) AP_MODE_SPECULATIVE (will be used in a future ap_getline rewrite) AP_MODE_EXHAUSTIVE (old *readbytes == -1 case) AP_MODE_INIT (special case for NNTP over SSL) The block parameter is an apr_read_type_e: APR_BLOCK_READ, APR_NONBLOCK_READ This also allows cleanup of mod_ssl's handling in the getline case. Reviewed by: Ryan Bloom (concept), Greg Stein (concept)
Fix seg fault in HTTP_IN when trying to handle the LimitRequestBody directive when reading a proxy response. We should bypass LimitRequestBody activities when handling a response from the proxied server. I don't think this will circumvent requestbody checking on a large request coming into the server that is destined to be proxied. (not tested though).
Change the return code from APR_TIMEUP to APR_ECONNABORTED, which seems to be a bit more descriptive. Move the check to inside the inner while() loop and add an additional check for eos. If we get an EOS bucket, there is no point in going further. Hopefully this will fix the last seg fault in the function.
Eliminate loop in ap_proxy_string_read(). Need to be able to handle getting an empty brigade from ap_get_brigade(). Also made sure that we always call ap_get_brigade() in readline mode. Submitted/Reviewed by: Adam Sussman & Bill Stoddard
Detabify some functions. No functional change.
make the proxy code spit out an error to the error log. This bit me when I included a 11th file no error message, nothing.
Hmmm... mismatched namespaces :-?
Cleanup the proxy code that creates a request to the origin server. This change adds an optional hook, which allows modules to gain control while the request is created if the proxy module is loaded. The purpose of this hook is to allow modules to add input and/or output filters to the request to the origin. While I was at it, I made the core use this hook, so that proxy request creation uses some of the code from the core. This can still be greatly improved, but this is a good start.
dechunking filter removed. add HTTP_IN filter after the headers have been parsed. so that the HTTP_IN can check them ;-) Still not 100% if this works properly, but I don't get the chunking numbers displayed on my browser, so it's better than what was there. PR: Obtained from: Submitted by: Reviewed by:
initial port of new Request-mode HTTP_IN filter to proxy. Status: It serves a reverse-proxy page.. but SLOWLY. VERY SLOWLY. still looking at why, but now it serves pages instead of dumping core. PR: Obtained from: Submitted by: Reviewed by:
change the parameter to ap_get_brigade from apr_site_t to apr_off_t, due to a parameter change in the original filter. PR: Submitted by: barbee@veribox.net Reviewed by: ianh@apache.org
inet_addr and inet_network are not interchangable. Furthermore, use apr_inet_addr, rather than ap_inet_addr (though they map to the same thing).
After checking what this does with Bill Rowe, and discovering it to be dead wood, I am tossing it.
Being slightly picky: properly namespace protect make_fake_req()
Nobody screamed. ;) This gets rid of ap_proxy_host2addr, and a few other useless functions, and allows us to be threadsafe when doing DNS resolution.
Reorganize ap_proxy_string_read() internally to not process eos buckets.
An EOS bucket doesn't contain anything, so why shouldn't we skip it?
Use apr-util's date functions now
Allows Mod_proxy to be dynamically loaded on win32 systems PR: Obtained from: Submitted by: ianh Reviewed by: Chuck Murcko, Graham
Crikey.
Track Greg's change to byte count for input filters
Made the variable naming the same as everywhere else in Apache for the ap_get_brigade() fix. PR: Obtained from: Reviewed by:
Just for the sake of cleanliness ...
Apply changes to ap_get_brigade() to the proxy code. PR: Obtained from: Reviewed by:
Make hooks work for the *_canon() functions. Work continues. PR: Obtained from: Reviewed by:
FTP directory filter works now. Many FIXME notes added. Much overhauling of proxy_ftp.c PR: Obtained from: Reviewed by:
Make sure the filter stack in a reused downstream connection is reset so we don't get lots of DECHUNK filters when we don't want them. PR: Obtained from: Reviewed by:
We now keep the entire conn_rec across downstream keepalives, not just the socket. Made the "keepalive mismatch" message more accurate. PR: Obtained from: Reviewed by:
Compiler warnings - yuck! Moved ap_proxy_string_read() to proxy_util.c so it can be used by proxy_http.c PR: Obtained from: Reviewed by:
Bugfixes to proxy_ftp.c -> in some places uri_addr was used instead of connect_addr - all hail cut and paste! PR: Obtained from: Submitted by: Reviewed by:
Converted send_dir() to ap_proxy_send_dir_filter() in proxy_ftp.c. Fixed up the header files PR: Obtained from: Submitted by: Reviewed by:
Cleanup of dead functions within proxy_util.c. PR: Obtained from: Reviewed by:
Some code rewriting in ap_proxy_connect_handler(): *) Fixed bug where a hostname without a "." in it (such as "localhost") would not trigger an IP address check with ProxyBlock. *) Fixed ProxyBlock bugs with ap_proxy_http_handler() and ap_proxy_connect_handler(). *) Updated ap_proxy_connect_handler() to support APR, while moving some common code between http_handler and connect_handler to proxy_util.c. PR: Obtained from: Reviewed by:
*) Added support for downstream keepalives in mod_proxy. *) Changed mod_proxy ap_proxy_http_handler() to support APR properly. PR: Obtained from: Reviewed by:
reorganize STATUS; update copyright
change INADDR_NONE to APR_INADDR_NONE Submitted by: Ian Holsman <IanH@cnet.com> Reviewed by: Chuck Murcko
Pull in the Connection handling part of the v1.3 patch for HTTP/1.1. Some preliminary HTTP/1.1 work Some comments and fixes PR: Obtained from: Reviewed by:
Cache removed, with other spooge PR: Obtained from: Submitted by: Reviewed by:
Fix any compile breaks after moving apr_snprintf and apr_vsnprintf to apr_strings.h
renaming various functions for consistency sake see: http://apr.apache.org/~dougm/apr_rename.pl PR: Obtained from: Submitted by: Reviewed by:
Changes to get running on the current codebase PR: Obtained from: Submitted by: Reviewed by:
Fix the remaining APR_FINFO_NORM occurances. Again, followup is needed to actually test any field outside the scope of APR_FINFO_MIN.
The changes required for the APR_FINFO_wanted argument to apr_stat/lstat/getfileinfo. These are -NOT- optimal, they are simply the required changes to get the server working. The size of the patch is a warning about how we need to really look at what we are trying to accomplish with all of these stat/lstat calls.
The big change. This is part 3 of the apr-util symbols rename, please see the first commit of srclib/apr-util/include (cvs apr-util/include) for the quick glance at symbols changed.
Switch to the APR-provided APR_CHARSET_EBCDIC feature test macro.
Force all Apache functions to be linked into the executable, whether they are used or not. This uses the same mechanism that is used for APR and APR-util. This may not be the correct solution, but it works, and that is what I really care about. This also renames CHARSET_EBCDIC to AP_CHARSET_EBCDIC. This is for namespace correctness, but it also makes the exports script a bit easier.
API routines ap_pgethostbyname() and ap_pduphostent() are no longer available. Use apr_getaddrinfo() instead. The ap_pduphostent() code was moved to modules/proxy/proxy_util.c for now since that is the only caller. When the proxy's use of the resolver is APR-ized this won't be needed anymore.
Restore Ryan's original intent in ap_proxy_doconnect() return value check
More proxy fixes. There are still some bugs in this code, but this has successfully proxied www.yahoo.com and www.ntrnet.net among other test sites that I chose at random.
fix apr_getaddrinfo() call
APR: Change apr_connect() to take apr_sockaddr_t instead of hostname. Add generic apr_create_socket(). Add apr_getaddrinfo() for doing hostname resolution/address string parsing and building apr_sockaddr_t. Submitted by: David Reid Reviewed by: Jeff Trawick
This removes all BUFF's from the HTTP proxy. This code is relatively ugly, but it does proxy pages. This even fixes the content-type bug that I introduced yesterday sometime. As soon as BUFF is removed from the FTP proxy, the buff.c and buff.h files need to go away.
Make the proxy work with filters. We are still using BUFF for the back-end communication, but BUFF has been removed from the communication with the client. The headers are not working correctly yet, but we are making progress. The most important part of this commit is that we are able to actually filter data coming from the proxy.
Simplify ap_proxy_doconnect(); now returns apr_status_t PR: Obtained from: Submitted by: Reviewed by:
The whole proxy compiles cleanly now. Now we need to make it actually work again, but at least we are making progress.
proxy_ftp and proxy_http both compile now. Not tested because the rest of the proxy is still not compiling, but we are making progress.
This is the 1.3.x current proxy into 2.0 - first stage PR: Obtained from: Sam Magnuson Submitted by: Chuck Murcko Reviewed by:
Not all platforms have INADDR_NONE defined by default. Apache used to check for this and the define it if needed. Since APR also needs this check it makes more sense for APR to just check and export a symobl that is always available. Submitted by: Branko �ibej <brane@xbc.nu> Reviewed by: Ryan Bloom
Missed it, a rather messy canonical error example.
Remove all compiler warnings from the proxy code. There are no garauntees that this still works, but most of the changes were just const-safeness changes. PR: 6631
Remaining cleanup of ap_ -> apr_ and AP_ -> APR_ transformation... see src/lib/apr/apr_compat.h for most details. Also a few minor nits to get Win32 to build. PR: Obtained from: Submitted by: Reviewed by:
prefix libapr functions and types with apr_
Add APR_EOL_STR for a platform specific text delimiter, provided by apr.h (defined in apr.h.in and apr.hw). This is needed -only- in APR created files (true raw files) such as logs. It is not required in any splat to screen (stderr/stdout) formatting, nor any html markup. Some other modules slipped through in the prior apr_strings.h commit. Sorry 'bout that. PR: Obtained from: Submitted by: Reviewed by:
fixups for remaining current warnings Submitted by: Chuck Murcko
New mod_proxy/mod_cache (file cache) for 2.0. This uses a caching API so that shared mem, LDAP servers, DBs, etc. can also be used for proxy caching. The caching API is very young, and subject to change as APR changes. proxy_cache.c from the proxy subdir is no longer used. Build with --enable-modules=proxy,cache Lightly tested on Linux, no warranties expressed or implied yet. Still to do: strerror(), waitpid(), stat() cleanups address issues marked by @@@FIXME add HTTP/1.1 This should be considered a *reference* proxy implementation for 2.0. What actually ends up shipping with 2.0 is likely going to be rather different as the redesign evolves. This may end up being the 2.0 backwards compatibility workalike. Modified to work with today's conf/build/layout scheme PR: Obtained from: Submitted by: Sam Magnuson Reviewed by: Chuck Murcko
back out APR_ prefix for TRUE,FALSE
prefix TRUE,FALSE with APR_
Finished move of ap_md5 routines to apr_md5. Removed ap_md5.h. Replaced more magic numbers with MD5_DIGESTSIZE. Yuck. Submitted by: William Rowe, Roy Fielding
Change ap_context_t to ap_pool_t. This compiles, runs, and serves pages on Linux, but probably breaks somewhere.
Update to Apache Software License version 1.1
Backport the CSS security fixes to Apache 2.0a. Or is that forward port? My sense of direction is all confused. PR: Obtained from: Submitted by: Reviewed by:
Fix all the License issues. Including: s/Apache Group/Apache Software Foundation/ s/1999/2000/ s/Sascha's license/ASF license
Finish the conversion from fprintf(stderr -> ap_log_err.
Remove most of the annoying "re ap_context_t mplement" mis-replacements by what I think is their real meaning. Guys, I didn't verify all of this, please have a close look at my "fixes". Thanks.
Fix all of those annoying --i -> - ap_context_t problems caused by a search/replace gone wrong.
Changed pools to contexts. Tested with prefork and pthread mpm's. I'll check this out tomorrow and make sure everything was checked in correctly.
Changes from pthreads. Removes timeouts, adds mutex. This probably breaks the proxy on everything but pthreads. Submitted by: Bill Stoddard
Apache 1.3.9 baseline for the Apache 2.0 repository. Obtained from: Apache 1.3.9 (minus unused files), tag APACHE_1_3_9 Submitted by: Apache Group
This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.
| infrastructure at apache.org | ViewVC Help |
| Powered by ViewVC 1.1.26 |