Harden dynamic HTTP and compiler boundaries

This commit is contained in:
root
2026-07-26 11:28:11 +00:00
parent 3d155203bd
commit d0efab7db0
30 changed files with 906 additions and 109 deletions
+1 -1
View File
@@ -8,7 +8,7 @@ return value : map with ok, bounded error code, and operation-specific output
:content
Runs one explicitly supported structured asymmetric cryptographic operation. The initial allowlist is `key_generate` with `ES256` and `jwt_sign` with `ES256`. Unknown operations and algorithms fail closed.
ES256 signing validates that `x`, `y`, and `d` form one P-256 key, forces the protected `alg` to `ES256`, and emits a compact JWT with a 64-byte JOSE signature. Requests, nesting, values, and output are bounded. Header and claims roots must be JSON objects containing valid UTF-8 without raw control bytes.
ES256 signing validates that `x`, `y`, and `d` form one P-256 key, forces the protected `alg` to `ES256`, and emits a compact JWT with a 64-byte JOSE signature. Requests are capped at 32 KiB; `cbor_decode` accepts at most 16 KiB decoded CBOR (21,846 canonical base64url characters), 256 nodes, and depth 16. CBOR is definite-length only, validates UTF-8 text, preserves typed map keys, rejects structurally duplicate keys and trailing bytes, and reports malformed CBOR as `invalid_cbor`. Header and claims roots must be JSON objects containing valid UTF-8 without raw control bytes.
This function does not replace typed digest, HMAC, password, randomness, or constant-time comparison APIs. It exposes no raw signing, arbitrary curve/digest selection, encryption, or generic OpenSSL access. Keep returned private JWKs secret.
+27 -2
View File
@@ -1,8 +1,18 @@
# http_request
Performs a bounded outbound HTTP(S) request using the runtime `curl` binary. Request fields: `method`, `url`, `headers`, `body`, `timeout_ms`, `follow_redirects`.
Performs an outbound HTTP(S) request using the runtime `curl` binary. Existing request fields remain backward-compatible: `method`, `url`, `headers`, `body`, `timeout_ms`, and `follow_redirects`.
Returns `{ status, headers, body, error }`. `headers` is a name/value map. A missing `curl` binary returns a clear `error` string.
An absent `security` map, or a map containing none of these recognized keys, keeps the legacy request behavior. If a `security` map contains any recognized key, it selects hardening and **every** recognized field below must be explicitly boolean `true`; missing, partial, non-true, or `false` fields fail closed with `invalid_request`.
- `https_only` rejects non-HTTPS URLs, IP-literal hosts, and URL userinfo.
- `public_dns_only` validates **every** DNS answer against the public IPv4 policy. IPv6 answers currently fail closed.
- `pin_dns` pins curl to one validated answer with `--resolve`, retaining the URL hostname for TLS/SNI.
- `isolated_curl` uses absolute `/usr/bin/curl`, `--disable` as argv[1], cleared environment, no proxy/config/netrc/HSTS/Alt-Svc inheritance, no redirects, a three-second connect bound, and a ten-second total bound.
- `no_redirects` makes redirect following and `follow_redirects=true` invalid composition.
Hardened requests bound body input/output to 64 KiB and response headers to 8 KiB. Async hardened requests keep curl and its descendants in the job worker process group, so cancelling that job kills that group only. Methods are limited to `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, and `OPTIONS`; header names/values are validated and caller-controlled `Host`, framing, connection, and expectation headers are rejected. Sensitive request bodies go to curl stdin, never argv. Errors are typed non-secret values such as `invalid_request`, `unsafe_dns`, `timeout`, `response_too_large`, `redirect_not_allowed`, `http_status`, and `network_failure`.
Returns `{ status, headers, body, error }`.
:see
>socket
@@ -12,3 +22,18 @@ DValue req; req["method"] = "GET"; req["url"] = "http://127.0.0.1/doc/index.uce"
req["headers"]["Host"] = "uce.openfu.com"; req["timeout_ms"] = (f64)2000;
DValue resp = http_request(req);
print("HTTP ", resp["status"].to_u64(), ", ", resp["body"].to_string().length(), " bytes returned\n");
:example
// GitHub token exchange: client_secret stays in stdin body, not argv.
DValue token; token["method"]="POST"; token["url"]="https://github.com/login/oauth/access_token";
token["headers"]["Accept"]="application/json"; token["headers"]["Content-Type"]="application/x-www-form-urlencoded";
token["body"]="client_id="+uri_encode(client_id)+"&client_secret="+uri_encode(client_secret)+"&code="+uri_encode(code);
for(String key:{"https_only","public_dns_only","pin_dns","isolated_curl","no_redirects"}) token["security"][key].set_bool(true);
DValue token_response=http_request(token);
:example
// ATProto metadata discovery uses the same generic policy; no provider operation name.
DValue meta; meta["method"]="GET"; meta["url"]="https://"+issuer_host+"/.well-known/oauth-authorization-server";
meta["headers"]["Accept"]="application/json";
for(String key:{"https_only","public_dns_only","pin_dns","isolated_curl","no_redirects"}) meta["security"][key].set_bool(true);
DValue metadata=http_request(meta);
+1 -1
View File
@@ -1,6 +1,6 @@
# http_request_async
Starts the same bounded curl-backed request as `http_request()` in the file-backed async job registry and returns a job id. Use `job_await()` or `job_result()` to retrieve the HTTP result.
Starts the same request shape as `http_request()`, including its opt-in `security` hardening fields, in the file-backed async job registry and returns a job id. An unknown `security` object remains legacy; any recognized hardening key requires all five keys to be explicit `true` or the request fails closed. Hardened curl remains in this job's worker process group, so `job_cancel()` kills curl and its descendants without signalling other jobs. Use `job_await()` or `job_result()` to retrieve the HTTP result.
:see
>socket
+4 -1
View File
@@ -16,6 +16,8 @@ Executes a MySQL query and returns the resulting data, if any.
`params` provides the query parameter values used by the statement. Use named `:name` placeholders only; positional `?` placeholders are rejected.
Ordinary placeholders are escaped and quoted as SQL string values. For grammar positions that require an unquoted non-negative integer, such as `LIMIT` and `OFFSET`, append `!` to the placeholder (`:limit!`). Unsigned placeholders fail before query execution unless their value is a non-empty sequence of decimal digits. They never accept signs, whitespace, expressions, identifiers, or other SQL fragments.
The result is returned as a `DValue`, which makes it easy to iterate through rows and read fields with the usual `DValue` accessors.
After an insert, update, or delete, use `mysql_affected_rows()` to inspect how many rows changed.
@@ -24,7 +26,8 @@ After an insert, update, or delete, use `mysql_affected_rows()` to inspect how m
MySQL* db = mysql_connect();
if(db != 0)
{
DValue rows = mysql_query(db, "select 'ada@example.test' as email, 1 + 1 as total");
StringMap params; params["limit"] = "1";
DValue rows = mysql_query(db, "select 'ada@example.test' as email, 1 + 1 as total limit :limit!", params);
String email = "none"; String total = "?";
rows.each([&](DValue r, String key) { email = r["email"].to_string(); total = r["total"].to_string(); });
print(email, " / total=", total, "\n");
+1 -1
View File
@@ -21,7 +21,7 @@ Starts a repeating background worker process.
If a process with the same `key` is already running anywhere in the runtime instance, `task_repeat()` does not start a second worker and instead returns the PID of the existing one. Coordination is through the same shared task state used by `task()`.
`timeout` bounds the lifetime of the repeating worker. The default is ten minutes. Pass `0` only for workers that have another shutdown path.
`timeout` bounds the lifetime of the repeating worker. The default is ten minutes. Pass `0` only for workers that have another shutdown path. In Wasm, each individual callback invocation remains bounded by the runtime invocation timeout even when the repeating process lifetime is unbounded.
:example
task_repeat("doc-demo-repeat", 60.0, []() { usleep(10000); });
+31
View File
@@ -20,6 +20,37 @@ CLI(Request& context)
print(cli_arg(context, "missing", "fallback"), "\n");
return;
}
if(action == "task_repeat_unbounded")
{
String marker = "/tmp/uce-task-repeat-unbounded.txt";
file_unlink(marker);
pid_t pid = task_repeat("uce-task-repeat-unbounded", 0.05, [marker]() { file_put_contents(marker, "ran"); }, 0);
usleep(200000);
bool ran = file_get_contents(marker) == "ran";
if(pid > 0) task_kill(pid, 15);
file_unlink(marker);
print(pid > 0 && ran ? "task repeat unbounded ok\n" : "task repeat unbounded failed\n");
return;
}
if(action == "mysql_unsigned_params")
{
MySQL db;
StringMap params; params["limit"] = "1";
String parsed = db.parse_query_parameters("SELECT 7 LIMIT :limit!", params);
StringMap comparison; comparison["left"] = "7"; comparison["right"] = "8";
String inequality = db.parse_query_parameters("SELECT :left!=:right", comparison);
StringMap empty_name; empty_name[""] = "1";
db.parameter_error = false; db.statement_info = "";
db.parse_query_parameters("SELECT 1 LIMIT :!", empty_name);
bool empty_rejected = db.parameter_error;
StringMap invalid; invalid["limit"] = "1;SELECT 9";
db.parameter_error = false;
db.statement_info = "";
db.parse_query_parameters("SELECT 1 LIMIT :limit!", invalid);
bool ok = parsed.find("LIMIT 1") != String::npos && inequality.find("'7'!='8'") != String::npos && empty_rejected && db.parameter_error && db.statement_info.find("must contain only decimal digits") != String::npos;
print(ok ? "mysql unsigned params ok\n" : "mysql unsigned params failed: parsed=" + parsed + ", parameter_error=" + (db.parameter_error ? "1" : "0") + ", statement=" + db.statement_info + "\n");
return;
}
context.set_status(400, "CLI Error");
print("unknown cli action: ", action, "\n");
}
+11 -3
View File
@@ -31,7 +31,7 @@ RENDER(Request& context)
StringMap colon_uri_headers = split_http_headers("GET /clock.uce?t=12:30 HTTP/1.1\r\nHost: colon.example\r\n");
check("split_http_headers() request line with colon in URI", colon_uri_headers["REQUEST_METHOD"] == "GET" && colon_uri_headers["DOCUMENT_URI"] == "/clock.uce" && colon_uri_headers["QUERY_STRING"] == "t=12:30" && colon_uri_headers["HTTP_HOST"] == "colon.example", var_dump(colon_uri_headers));
check("trim() / split_kv() / split_http_headers()", trim(" padded value ") == "padded value" && kv["alpha"] == "one" && kv["empty"] == "" && http_headers["REQUEST_METHOD"] == "GET" && http_headers["DOCUMENT_URI"] == "/demo.uce" && http_headers["QUERY_STRING"] == "x=1" && http_headers["HTTP_X_EMPTY"] == "" && leading_crlf_headers["REQUEST_METHOD"] == "GET" && leading_crlf_headers["DOCUMENT_URI"] == "/lead.uce" && leading_crlf_headers["HTTP_HOST"] == "lead.example" && header_only["REQUEST_METHOD"] == "" && header_only["HTTP_HOST"] == "example.test" && header_only["HTTP_X_TOKEN"] == "abc", trim(" padded value ") + " / " + var_dump(kv) + " / " + var_dump(http_headers) + " / " + var_dump(leading_crlf_headers) + " / " + var_dump(header_only));
check("replace()", replace("alpha-beta-beta", "beta", "done") == "alpha-done-done", replace("alpha-beta-beta", "beta", "done"));
check("replace()", replace("alpha-beta-beta", "beta", "done") == "alpha-done-done" && replace("hello", "", "X") == "hello", replace("alpha-beta-beta", "beta", "done"));
check("html_escape() attribute-safe quotes", html_escape("<&>\"Don't") == "&lt;&amp;&gt;&quot;Don&#39;t", html_escape("<&>\"Don't"));
check("regex_match()", regex_match("[A-Z][a-z]+", "Alice") && !regex_match("[A-Z][a-z]+", "Alice!"), "full-string validation");
@@ -212,11 +212,19 @@ RENDER(Request& context)
String decoded_base64 = base64_decode(encoded_base64, base64_ok);
bool invalid_base64_ok = true;
base64_decode("AA=A", invalid_base64_ok);
check("base64_encode() / base64_decode() binary-safe", base64_ok && decoded_base64 == binary_payload && decoded_base64.size() == binary_payload.size() && !invalid_base64_ok, encoded_base64 + " bytes=" + std::to_string((u64)decoded_base64.size()));
bool early_padding_base64_ok = true;
base64_decode("A=AA", early_padding_base64_ok);
check("base64_encode() / base64_decode() binary-safe", base64_ok && decoded_base64 == binary_payload && decoded_base64.size() == binary_payload.size() && !invalid_base64_ok && !early_padding_base64_ok, encoded_base64 + " bytes=" + std::to_string((u64)decoded_base64.size()));
String utf8_sample = "A\xC3\xA9";
auto utf8_parts = split_utf8(utf8_sample);
check("split_utf8()", utf8_parts.size() == 2, "count=" + std::to_string(utf8_parts.size()));
String truncated_utf8 = "abc";
truncated_utf8.push_back((char)0xE2);
truncated_utf8.push_back((char)0x82);
auto truncated_utf8_parts = split_utf8(truncated_utf8);
String lone_utf8_lead(1, (char)0xF0);
auto compound_lone_utf8_parts = split_utf8(lone_utf8_lead, true);
check("split_utf8()", utf8_parts.size() == 2 && truncated_utf8_parts.size() == 4 && truncated_utf8_parts[3].size() == 2 && compound_lone_utf8_parts.size() == 1 && compound_lone_utf8_parts[0] == lone_utf8_lead, "count=" + std::to_string(utf8_parts.size()) + " truncated=" + std::to_string(truncated_utf8_parts.size()));
DValue payload;
payload["name"] = "uce";
+17 -3
View File
@@ -41,6 +41,17 @@ RENDER(Request& context)
wrong_request["private_jwk"] = wrong_curve;
DValue malformed_request = sign_request;
malformed_request["private_jwk"] = malformed;
DValue cose_request;
cose_request["operation"] = "cose_es256_parse";
cose_request["algorithm"] = "ES256";
cose_request["cose_key_base64url"] = "pQECAyYgASFYIGsX0fLhLEJH-Lzm5WOkQPJ3A32BLeszoPShOUXYmMKWIlggT-NC4v4af5uO5-tKfA-eFivOM1drMV7Oy7ZAaDe_UfU";
DValue cose = crypto_operation(cose_request);
DValue verify_request;
verify_request["operation"] = "es256_verify";
verify_request["algorithm"] = "ES256";
verify_request["cose_key_base64url"] = cose_request["cose_key_base64url"];
verify_request["message_base64url"] = "d2ViYXV0aG4gZml4ZWQgbWVzc2FnZQ";
verify_request["signature_der_base64url"] = "MEUCIQCkatZK1VVsjk17uvyzyhjdAkMNWXPjxSOMqWcjmM_8XAIgDaSk3Qufyd0_6r9Dm9A8RQbFco-FdTBulq7bvRGoBC4";
DValue unsupported;
unsupported["operation"] = "encrypt";
unsupported["algorithm"] = "ES256";
@@ -53,9 +64,12 @@ RENDER(Request& context)
list_request["protected_header"] = list_header;
DValue control_request = sign_request;
control_request["claims"]["bad"] = String("control\nbyte");
DValue oversized = key_request;
oversized["ignored"] = String(17000, 'x');
check("crypto_operation() ES256 key generation and JWT signing", key["ok"].to_bool() && signed_result["ok"].to_bool() && key["private_jwk"]["kty"].to_string() == "EC" && key["private_jwk"]["crv"].to_string() == "P-256" && key["public_jwk"]["d"].to_string() == "" && key["kid"].to_string() == key["thumbprint"].to_string() && parts.size() == 3 && parts[2].size() == 86 && decoded_header.find("ES256") != String::npos && !crypto_operation(wrong_request)["ok"].to_bool() && !crypto_operation(malformed_request)["ok"].to_bool() && crypto_operation(unsupported)["error"].to_string() == "unsupported_operation" && !crypto_operation(list_request)["ok"].to_bool() && crypto_operation(control_request)["error"].to_string() == "invalid_request" && crypto_operation(oversized)["error"].to_string() == "invalid_request", key["kid"].to_string());
check("crypto_operation() generates an ES256 key", key["ok"].to_bool() && key["private_jwk"]["kty"].to_string() == "EC" && key["private_jwk"]["crv"].to_string() == "P-256" && key["public_jwk"]["d"].to_string() == "" && key["kid"].to_string() == key["thumbprint"].to_string(), key["kid"].to_string());
check("crypto_operation() signs ES256 JWTs", signed_result["ok"].to_bool() && parts.size() == 3 && parts[2].size() == 86 && decoded_header.find("ES256") != String::npos, signed_result["error"].to_string());
check("crypto_operation() rejects invalid signing requests", !crypto_operation(wrong_request)["ok"].to_bool() && !crypto_operation(malformed_request)["ok"].to_bool() && !crypto_operation(list_request)["ok"].to_bool() && crypto_operation(control_request)["error"].to_string() == "invalid_request", "signing validation");
check("crypto_operation() parses an ES256 COSE key", cose["ok"].to_bool(), cose["error"].to_string());
check("crypto_operation() verifies ES256 DER signatures", crypto_operation(verify_request)["ok"].to_bool() && crypto_operation(verify_request)["valid"].to_bool(), crypto_operation(verify_request)["error"].to_string());
check("crypto_operation() rejects unsupported operations", crypto_operation(unsupported)["error"].to_string() == "unsupported_operation", crypto_operation(unsupported)["error"].to_string());
site_tests_summary(passed, failed, skipped, "Structured crypto tests generate ephemeral P-256 keys and retain no key material.");
site_tests_page_end();
}
+4
View File
@@ -31,6 +31,8 @@ RENDER(Request& context)
};
String nested = preprocessor_nested_literal();
String trailing_backslash = "C:\\Users\\";
String escaped_quote = "before\"after";
site_tests_page_start("Preprocessor", "Regression coverage for literal output rewriting and parser edge cases.");
?>
@@ -43,6 +45,8 @@ RENDER(Request& context)
check("raw string terminator in nested literal", contains(nested, "nested )\" marker"), nested);
check("entrypoint @fragment attribute captures output", context.call["fragments"]["preprocessor-test"].to_string() == "fragment attr once", context.call["fragments"]["preprocessor-test"].to_string());
check("inline code island after dangerous literal", true, "parser returned to C++ after rendering literal content containing )\"");
check("quote scanner handles trailing escaped backslash", trailing_backslash == "C:\\Users\\", trailing_backslash);
check("quote scanner retains escaped quote content", escaped_quote == "before\"after", escaped_quote);
site_tests_summary(passed, failed, skipped, "Literal content containing the C++ raw-string terminator sequence must compile and render unchanged.");
site_tests_page_end();
+26 -3
View File
@@ -28,14 +28,20 @@ RENDER(Request& context)
u64 sockfd = socket_connect("127.0.0.1", 80);
if(sockfd != 0)
{
u64 closed_handle = socket_connect("127.0.0.1", 80);
bool opaque_handles = sockfd == 1 && closed_handle == 2;
socket_close(closed_handle);
socket_close(closed_handle);
socket_close(999999);
bool closed_write_rejected = !socket_write(closed_handle, "invalid");
bool write_ok = socket_write(sockfd, "GET /tests/index.uce HTTP/1.0\r\nHost: uce.openfu.com\r\n\r\n");
String response = socket_read(sockfd, 4096, 2);
socket_close(sockfd);
bool has_nul = response.find(String("\0", 1)) != String::npos;
mark(
"socket_connect() / socket_write() / socket_read()",
(write_ok && response.find("200 OK") != String::npos && !has_nul) ? "pass" : "fail",
response.substr(0, response.length() > 220 ? 220 : response.length()) + (has_nul ? " [unexpected NUL]" : "")
"socket handles reject invalid and closed indices",
(opaque_handles && closed_write_rejected && write_ok && response.find("200 OK") != String::npos && !has_nul) ? "pass" : "fail",
"handles=" + std::to_string(sockfd) + "/" + std::to_string(closed_handle) + " " + response.substr(0, response.length() > 180 ? 180 : response.length()) + (has_nul ? " [unexpected NUL]" : "")
);
}
else
@@ -130,6 +136,15 @@ RENDER(Request& context)
parsed_underscore.find("'Ada'") != String::npos && parsed_underscore.find("_name") == String::npos ? "pass" : "fail",
parsed_underscore
);
StringMap quoted_params;
quoted_params["name_looking_text"] = "SUBSTITUTED";
String escaped_quote_query = "SELECT 'a\\':name_looking_text'";
String parsed_escaped_quote = placeholder_guard.parse_query_parameters(escaped_quote_query, quoted_params);
mark(
"mysql named placeholders stay opaque after escaped quotes",
parsed_escaped_quote.find(":name_looking_text") != String::npos && parsed_escaped_quote.find("SUBSTITUTED") == String::npos ? "pass" : "fail",
parsed_escaped_quote
);
MySQL unavailable_mysql;
unavailable_mysql.connect("127.0.0.1", "__uce_intentionally_missing__", "not-a-real-password");
mark(
@@ -173,6 +188,14 @@ RENDER(Request& context)
inserted == 2 && updated == 1 && selected == 0 ? "pass" : "fail",
"inserted=" + std::to_string((u64)inserted) + " updated=" + std::to_string((u64)updated) + " selected=" + std::to_string((u64)selected)
);
mysql.query("INSERT INTO uce_affected_rows_test VALUES (1,30)");
u32 duplicate_error_code = mysql._preload_next_error_code;
String duplicate_error = mysql.error();
mark(
"mysql query failures retain server diagnostics",
duplicate_error_code > 1 && duplicate_error != "" && duplicate_error != "Unknown server error" ? "pass" : "fail",
"code=" + std::to_string((u64)duplicate_error_code) + " error=" + duplicate_error
);
}
DValue mysql_perf = request_perf();
String mysql_operations = json_encode(mysql_perf["mysql_operations"]);
+5
View File
@@ -18,6 +18,10 @@ RENDER(Request& context)
auto unit_paths = units_list();
DValue info = unit_info("call_helpers.uce");
DValue relative_info = unit_info("components/../relative-child.uce");
String outside_unit = "/tmp/uce-site-tests-outside-unit.uce";
file_put_contents(outside_unit, "RENDER(Request& context) {}");
DValue outside_info = unit_info(outside_unit);
file_unlink(outside_unit);
ob_start();
unit_call("call_helpers.uce", "emit_marker");
@@ -32,6 +36,7 @@ RENDER(Request& context)
check("units_list()", unit_paths.size() > 0, "count=" + std::to_string(unit_paths.size()));
check("unit_info()", info["path"].to_string() != "", json_encode(info));
check("compiler canonicalizes relative unit paths", relative_info["path"].to_string().find("/../") == String::npos && str_ends_with(relative_info["path"].to_string(), "/site/tests/relative-child.uce"), json_encode(relative_info));
check("compiler rejects units outside document root", outside_info["path"].to_string() == "", json_encode(outside_info));
check("unit_compile()", unit_compile("call_helpers.uce"), "call_helpers.uce");
check("unit_call()", call_output.find("UNIT_CALL_EXPORT_OK") != String::npos, call_output);
check("unit_render()", render_output.find("data-unit-render=\"ok\"") != String::npos, render_output);