changing doc format and HTML literals

README.md

@@ -90,7 +90,13 @@ Those are intended for sub-rendering through helpers such as `component("compone
 
 ## Template Output
 
-Inside `<> ... </>` literal blocks, UCE supports three inline forms:
+UCE now treats template parsing as one shared code-vs-literal state machine.
+
+- `<>` and `?>` both enter literal output mode
+- `</>` and `<?` both return to code mode
+- the delimiter pairs are interchangeable, so either style can be used consistently or mixed locally
+
+Inside literal output, UCE supports three inline forms:
 
 - `<? ... ?>` to emit raw C++ statements
 - `<?= expression ?>` to print HTML-escaped output
@@ -98,7 +104,7 @@ Inside `<> ... </>` literal blocks, UCE supports three inline forms:
 
 Use `<?= ... ?>` by default for user-visible text. Use `<?: ... ?>` only for trusted markup or content that has already been escaped.
 
-The parser now treats C++ `//` and `/* ... */` comments as comments in both normal code and `<? ... ?>` islands, so quotes or `<>` markers inside comments do not confuse template parsing.
+The parser now treats C++ `//` and `/* ... */` comments as comments in both normal code and `<? ... ?>` islands, so quotes or delimiter markers inside comments do not confuse template parsing.
 
 The preprocessing implementation is now split between `src/lib/compiler.cpp` and `src/lib/compiler-parser.cpp`. `compiler.cpp` owns unit compilation and cache orchestration, while `compiler-parser.cpp` owns source rewriting and template parsing.
 

site/doc/pages/3_C++ Preprocessor.txt

@@ -14,12 +14,14 @@ unit_call
 :content
 UCE runs a small custom source-to-source preprocessor before Clang sees a `.uce` or `.ws.uce` file.
 
-The implementation lives in `src/lib/compiler.cpp`. It does not try to parse all of C++. Instead, it performs a narrow character-wise rewrite that understands UCE literal blocks, inline code islands, `#load`, and `EXPORT` harvesting, then writes a generated `.cpp` file and compiles that file into a shared object.
+The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, with orchestration in `src/lib/compiler.cpp`. It does not try to parse all of C++. Instead, it performs a narrow character-wise rewrite that understands literal output, inline code islands, `#load`, and `EXPORT` harvesting, then writes a generated `.cpp` file and compiles that file into a shared object.
 
 ## Syntax
 
 - `<> ... </>` enters literal-output mode.
-- Inside a literal block, `<? ... ?>` emits raw C++.
+- `?> ... <?` also enters literal-output mode.
+- The open and close pairs are interchangeable, so `<> ... <?`, `?> ... </>`, and the traditional matched forms all work.
+- Inside literal output, `<? ... ?>` emits raw C++.
 - Inside a literal block, `<?= expression ?>` emits `print(html_escape(expression));`.
 - Inside a literal block, `<?: expression ?>` emits `print(expression);` without HTML escaping.
 - `#load "other.uce"` injects another UCE unit at compile time.
@@ -32,7 +34,9 @@ The implementation lives in `src/lib/compiler.cpp`. It does not try to parse all
 - The generated file starts by including `COMPILER_SYS_PATH/src/lib/uce_lib.h`.
 - It then inlines the configured setup template from `SETUP_TEMPLATE` (by default `scripts/setup.h.template`), which defines the internal hook `__uce_set_current_request(Request*)`.
 - It inserts `#line 1` before page code so compiler diagnostics point back to the original `.uce` file.
-- Each literal block is rewritten into one or more `print(R"( ... )");` calls.
+- Each literal region is rewritten into one or more `print(R"( ... )");` calls.
+- `<>` and `?>` both switch from code mode into literal output.
+- `</>` and `<?` both switch from literal output back into code mode.
 - `<? ... ?>` temporarily breaks out of literal printing, emits the enclosed C++ unchanged, then resumes literal output.
 - `<?= ... ?>` becomes `print(html_escape(...));`.
 - `<?: ... ?>` becomes `print(...);` and is intended for trusted markup or already-escaped content.
@@ -62,6 +66,15 @@ RENDER(Request& context)
 }
 ```
 
+The same thing can also be written with PHP-style literal delimiters:
+
+```cpp
+RENDER(Request& context)
+{
+    ?><h1><?= context.params["DOCUMENT_URI"] ?></h1><?
+}
+```
+
 Roughly becomes:
 
 ```cpp
@@ -102,8 +115,9 @@ The loaded file is resolved relative to the current source file unless the path
 
 ## Rules
 
-- Literal mode starts only on the exact token `<>`.
-- Literal mode ends only on the exact token `</>`.
+- Literal mode can start on either `<>` or `?>`.
+- Literal mode can end on either `</>` or `<?`.
+- Literal delimiters are interchangeable; the parser now treats them as one shared code-vs-literal state machine rather than as separate nested block types.
 - `#load` is recognized only when the current line starts with `#load ` at column 1.
 - `EXPORT` harvesting only triggers when the current line starts with `EXPORT` at column 1 and is followed by whitespace.
 - Relative `#load` paths are expanded against the including unit's source directory.
@@ -112,10 +126,9 @@ The loaded file is resolved relative to the current source file unless the path
 ## Limitations
 
 - This pass is character-wise, not a full parser.
-- Outside literal blocks it only tracks double-quoted C++ strings while deciding whether `<>` should open literal mode.
+- Outside literal blocks it tracks C++ quotes and comments while deciding whether `<>` or `?>` should open literal mode.
 - It does not understand comments, raw string literals, templates, or general C++ token structure.
-- Inside literal blocks it tracks single and double quotes while scanning a `<? ... ?>` island so quoted `?>` text does not close the island early.
-- Literal blocks are not nested.
+- Inside literal blocks it tracks quotes and comments while scanning `<? ... ?>`, `<?= ... ?>`, and `<?: ... ?>` islands so quoted `?>` text does not close those islands early.
 - Because literal output is emitted as a C++ raw string literal `R"( ... )"`, literal content must not contain the exact terminator sequence `)"` or the generated C++ will break.
 - `#load` depends on the target unit's generated `.cpp` existing and being compilable. If the target cannot be preprocessed or compiled correctly, the including file will fail to compile as well.
 

site/test/demo_guard.h

@@ -0,0 +1,88 @@
+String test_demo_normalize_ip(String ip)
+{
+	if(ip.find(",") != String::npos)
+		ip = trim(nibble(ip, ","));
+	ip = trim(ip);
+	if(str_starts_with(ip, "::ffff:"))
+		ip = ip.substr(7);
+	return(ip);
+}
+
+bool test_demo_ip_is_private(String ip)
+{
+	ip = trim(ip);
+	if(ip == "" || ip == "localhost" || ip == "::1")
+		return(true);
+	if(str_starts_with(ip, "127."))
+		return(true);
+	if(str_starts_with(ip, "10."))
+		return(true);
+	if(str_starts_with(ip, "192.168."))
+		return(true);
+	if(str_starts_with(ip, "fc") || str_starts_with(ip, "fd") || str_starts_with(ip, "fe80:"))
+		return(true);
+
+	auto parts = split(ip, ".");
+	if(parts.size() == 4 && parts[0] == "172")
+	{
+		s64 second = int_val(parts[1]);
+		if(second >= 16 && second <= 31)
+			return(true);
+	}
+
+	return(false);
+}
+
+String test_demo_request_ip(Request& context)
+{
+	String remote_ip = test_demo_normalize_ip(context.params["REMOTE_ADDR"]);
+	String forwarded_ip = test_demo_normalize_ip(first(context.params["HTTP_X_FORWARDED_FOR"], context.params["HTTP_X_REAL_IP"]));
+
+	if(remote_ip != "" && !test_demo_ip_is_private(remote_ip))
+		return(remote_ip);
+	if(forwarded_ip != "")
+		return(forwarded_ip);
+	return(remote_ip);
+}
+
+bool test_demo_request_allowed(Request& context)
+{
+	return(test_demo_ip_is_private(test_demo_request_ip(context)));
+}
+
+void test_demo_render_restricted_html(Request& context, String title, String risk, String style_href = "style.css", String back_href = "index.uce")
+{
+	context.set_status(403, "Restricted");
+	String request_ip = first(test_demo_request_ip(context), "unknown");
+	print("<html><head>");
+	print("<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\"></meta>");
+	print("<link rel=\"stylesheet\" href='", html_escape(style_href), "?v=", time(), "'></link>");
+	print("</head><body>");
+	print("<h1><a href=\"", html_escape(back_href), "\">UCE Test</a>: ", html_escape(title), "</h1>");
+	print("<p>This test page is disabled for public access because it can ", html_escape(risk), ".</p>");
+	print("<p>It remains available from localhost or a private network for local development and server administration.</p>");
+	print("<p>Detected request source: <code>", html_escape(request_ip), "</code></p>");
+	print("</body></html>");
+}
+
+void test_demo_render_restricted_text(Request& context, String title, String risk)
+{
+	context.set_status(403, "Restricted");
+	context.header["Content-Type"] = "text/plain; charset=utf-8";
+	print(title, ": restricted on public access\n");
+	print("Reason: this test can ", risk, ".\n");
+	print("Detected request source: ", first(test_demo_request_ip(context), "unknown"), "\n");
+}
+
+void test_demo_render_restricted_json(Request& context, String title, String risk)
+{
+	context.set_status(403, "Restricted");
+	context.header["Content-Type"] = "application/json; charset=utf-8";
+	DTree payload;
+	payload["ok"].set_bool(false);
+	payload["error"] = "restricted";
+	payload["title"] = title;
+	payload["reason"] = "This test is disabled for public access because it can " + risk + ".";
+	payload["request_ip"] = first(test_demo_request_ip(context), "unknown");
+	print(json_encode(payload));
+}

site/test/error-reporting.uce

@@ -1,6 +1,13 @@
+#include "demo_guard.h"
+
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "Error reporting", "intentionally crash or abort request workers");
+		return;
+	}
 	String mode = context.get["mode"];
 
 	if(mode == "exception")

site/test/file_append.uce

@@ -1,9 +1,16 @@
 
+#include "demo_guard.h"
+
 
 
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "File Append", "write to server-side files");
+		return;
+	}
 	DTree t;
 
 	<>

site/test/index.uce

@@ -1,4 +1,6 @@
 
+#include "demo_guard.h"
+
 
 void render_card(String url, String title, String desc)
 {
@@ -12,6 +14,7 @@ RENDER(Request& context)
 {
 	DTree p;
 	p.set(context.params);
+	bool allow_server_demos = test_demo_request_allowed(context);
 
 	<><html>
 	<head>
@@ -26,9 +29,9 @@ RENDER(Request& context)
 		<div class="test-grid">
 			<div class="grid-heading">Basics</div>
 			<? render_card("hello.uce", "Hello World", "Basic output and server time"); ?>
-			<? render_card("working-dir.uce", "Working Directory", "Server paths and cwd"); ?>
 			<? render_card("header.uce", "Headers", "HTTP response headers"); ?>
-			<? render_card("error-reporting.uce", "Error Reporting", "Error handling and output"); ?>
+			<? if(allow_server_demos) { render_card("working-dir.uce", "Working Directory", "Server paths and cwd"); } ?>
+			<? if(allow_server_demos) { render_card("error-reporting.uce", "Error Reporting", "Error handling and output"); } ?>
 
 			<div class="grid-heading">Data Types &amp; Parsing</div>
 			<? render_card("dtree.uce", "DTree", "Dynamic hierarchical data tree"); ?>
@@ -49,21 +52,27 @@ RENDER(Request& context)
 
 			<div class="grid-heading">Storage &amp; I/O</div>
 			<? render_card("fileio.uce", "File I/O", "Read and write files"); ?>
-			<? render_card("file_append.uce", "File Append", "Append data to files"); ?>
-			<? render_card("shell.uce", "Shell", "Execute shell commands"); ?>
-			<? render_card("memcached.uce", "Memcached", "Memcached key-value store"); ?>
-			<? render_card("mysql.uce", "MySQL", "MySQL database connector"); ?>
+			<? if(allow_server_demos) { render_card("file_append.uce", "File Append", "Append data to files"); } ?>
+			<? if(allow_server_demos) { render_card("shell.uce", "Shell", "Execute shell commands"); } ?>
+			<? if(allow_server_demos) { render_card("memcached.uce", "Memcached", "Memcached key-value store"); } ?>
+			<? if(allow_server_demos) { render_card("mysql.uce", "MySQL", "MySQL database connector"); } ?>
 
 			<div class="grid-heading">Advanced</div>
 			<? render_card("call_file.uce", "unit_call()", "Dynamic file inclusion"); ?>
-			<? render_card("unit-browser.uce", "Unit Browser", "units_list(), unit_info(), unit_compile()"); ?>
 			<? render_card("components.uce", "Components", "Reusable component system"); ?>
 			<? render_card("markdown.uce", "Markdown", "Markdown parsing with components"); ?>
 			<? render_card("script.uce", "Script", "UCE script integration"); ?>
-			<? render_card("task.uce", "Task", "Background task execution"); ?>
-			<? render_card("task_repeat.uce", "Task Repeat", "Recurring task scheduling"); ?>
 			<? render_card("websockets.ws.uce", "WebSockets", "Real-time WebSocket chat"); ?>
+			<? render_card("unit-browser.uce", "Unit Browser", "units_list(), unit_info(), unit_compile()"); ?>
+			<? if(allow_server_demos) { render_card("task.uce", "Task", "Background task execution"); } ?>
+			<? if(allow_server_demos) { render_card("task_repeat.uce", "Task Repeat", "Recurring task scheduling"); } ?>
 		</div>
+		<? if(!allow_server_demos) { ?>
+		<div class="system-info">
+			<h3>Restricted On Public Access</h3>
+			<pre>Local-only demos are hidden here because they can execute shell commands, access infrastructure services, mutate server state, compile units, or expose internal runtime details.</pre>
+		</div>
+		<? } else { ?>
 		<div class="system-info">
 			<h3>System Info</h3>
 			<pre><?
@@ -73,6 +82,7 @@ RENDER(Request& context)
 				print("Request #", context.server->request_count, "\n");
 			?></pre>
 		</div>
+		<? } ?>
 		<details>
 			<summary>Request Parameters</summary>
 			<pre><?= (var_dump(p)) ?></pre>

site/test/memcached.uce

@@ -1,7 +1,14 @@
 
+#include "demo_guard.h"
+
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "MemcacheD", "inspect and mutate local memcached state");
+		return;
+	}
 	DTree t;
 
 	<>

site/test/mysql.uce

@@ -1,7 +1,14 @@
 
+#include "demo_guard.h"
+
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "MySQL Connector Test", "connect to local database services and expose database contents");
+		return;
+	}
 
 	<><html>
 		<link rel="stylesheet" href='style.css?v=1'></link>

site/test/shell.uce

@@ -1,7 +1,14 @@
 
+#include "demo_guard.h"
+
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "Shell Stuff", "execute shell commands and expose repository state");
+		return;
+	}
 
 	<><html>
 		<link rel="stylesheet" href='style.css?v=1'></link>

site/test/task-status.uce

@@ -1,6 +1,13 @@
+#include "demo_guard.h"
+
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_text(context, "Task Status", "inspect background task state");
+		return;
+	}
 	String task_name = first(context.get["task-name"], "example-task");
 
 	print("Task Name: ", task_name, "\n");

site/test/task.uce

@@ -1,6 +1,13 @@
 
+#include "demo_guard.h"
+
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "Tasks", "spawn and inspect background tasks");
+		return;
+	}
 	DTree t;
 
 	String task_name = first(context.get["task-name"], "example-task");

site/test/task_repeat.uce

@@ -1,6 +1,13 @@
 
+#include "demo_guard.h"
+
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "Task repeat", "schedule recurring background tasks");
+		return;
+	}
 	DTree t;
 
 	String task_name = first(context.get["task-name"], "example-task");

site/test/test2/working-dir-test.uce

@@ -1,6 +1,13 @@
 
+#include "../demo_guard.h"
+
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_text(context, "Working Directory Sub-Invoke", "reveal internal filesystem layout");
+		return;
+	}
 	print("Sub-Invoke Working dir: ", cwd_get(), "\n");
 }

site/test/unit-browser.uce

@@ -1,3 +1,5 @@
+#include "demo_guard.h"
+
 String unit_query(String selected_path, String compile_path = "")
 {
 	String result = "?";
@@ -32,6 +34,7 @@ String unit_flag_label(DTree value)
 
 RENDER(Request& context)
 {
+	bool allow_mutation = test_demo_request_allowed(context);
 	String compile_target = first(context.get["compile"], "");
 	String selected_path = first(context.get["selected"], "");
 	String compile_message = "";
@@ -39,10 +42,19 @@ RENDER(Request& context)
 
 	if(compile_target != "")
 	{
-		bool compile_ok = unit_compile(compile_target);
 		selected_path = compile_target;
-		compile_message = (compile_ok ? "Compile succeeded for " : "Compile failed for ") + compile_target;
-		compile_message_class = compile_ok ? "status-ok" : "status-error";
+		if(allow_mutation)
+		{
+			bool compile_ok = unit_compile(compile_target);
+			compile_message = (compile_ok ? "Compile succeeded for " : "Compile failed for ") + compile_target;
+			compile_message_class = compile_ok ? "status-ok" : "status-error";
+		}
+		else
+		{
+			context.set_status(403, "Restricted");
+			compile_message = "Manual compile is restricted to localhost and private-network access.";
+			compile_message_class = "status-error";
+		}
 	}
 
 	auto unit_paths = units_list();
@@ -122,7 +134,11 @@ RENDER(Request& context)
 							</div>
 							<div class="unit-actions">
 								<span class="status-badge <?= selected_status_class ?>"><?= selected_compile_status ?></span>
-								<a href="<?= selected_compile_link ?>">compile</a>
+								<? if(allow_mutation) { ?>
+									<a href="<?= selected_compile_link ?>">compile</a>
+								<? } else { ?>
+									<span class="dim">compile disabled on public access</span>
+								<? } ?>
 							</div>
 						</div>
 

site/test/unit-info.uce

@@ -1,9 +1,21 @@
+#include "demo_guard.h"
+
 RENDER(Request& context)
 {
+	bool allow_mutation = test_demo_request_allowed(context);
 	DTree payload;
 
 	if(context.get["compile"] != "")
-		payload["compile_ok"].set_bool(unit_compile(context.get["compile"]));
+	{
+		if(allow_mutation)
+			payload["compile_ok"].set_bool(unit_compile(context.get["compile"]));
+		else
+		{
+			context.set_status(403, "Restricted");
+			payload["error"] = "restricted";
+			payload["reason"] = "Manual compile is restricted to localhost and private-network access.";
+		}
+	}
 
 	payload["current"] = unit_info();
 

site/test/working-dir.uce

@@ -1,7 +1,14 @@
 
+#include "demo_guard.h"
+
 
 RENDER(Request& context)
 {
+	if(!test_demo_request_allowed(context))
+	{
+		test_demo_render_restricted_html(context, "Working Directory", "reveal internal filesystem layout");
+		return;
+	}
 
 	<>
 		<link rel="stylesheet" href='style.css'></link>

src/lib/compiler-parser.cpp

@@ -145,6 +145,37 @@ String compiler_capture_markup_literal(const String& content, u32& i)
 	return(buffer);
 }
 
+String compiler_capture_code_until_php_close(const String& content, u32& i)
+{
+	String buffer = "";
+	CompilerCodeState code_state;
+
+	while(i < content.length())
+	{
+		char c = content[i];
+		char c1 = (i + 1 < content.length()) ? content[i + 1] : '\0';
+
+		if(compiler_code_state_is_neutral(code_state) && c == '?' && c1 == '>')
+		{
+			i += 1;
+			return(buffer);
+		}
+
+		compiler_code_state_consume(code_state, buffer, content, i);
+		i += 1;
+	}
+
+	return(buffer);
+}
+
+void compiler_append_text_literal_output(String& parsed_content, String& literal_buffer)
+{
+	if(literal_buffer == "")
+		return;
+	parsed_content.append("print(R\"(" + literal_buffer + ")\");");
+	literal_buffer.clear();
+}
+
 String compiler_process_text_literal(Request* context, SharedUnit* su, String content)
 {
 	String parsed_content;
@@ -314,22 +345,62 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
 		"#line 1\n";
 	CompilerCodeState code_state;
 	String current_line = "";
+	String literal_buffer = "";
+	bool inside_literal = false;
 
 	for(u32 i = 0; i < content.length(); i++)
 	{
 		char c = content[i];
 		char c1 = (i + 1 < content.length()) ? content[i + 1] : '\0';
-		current_line.append(1, c);
+		char c2 = (i + 2 < content.length()) ? content[i + 2] : '\0';
 
-		if(compiler_code_state_is_neutral(code_state) && c == '<' && c1 == '>')
+		if(inside_literal)
 		{
-			String markup_literal = compiler_capture_markup_literal(content, i);
-			parsed_content.append(compiler_process_text_literal(context, su, markup_literal));
-			if(i < content.length() && content[i] == '\n')
-				current_line = "\n";
+			if(c == '<' && c1 == '?' && (c2 == '=' || c2 == ':'))
+			{
+				compiler_append_text_literal_output(parsed_content, literal_buffer);
+				bool escape_field = (c2 == '=');
+				i += 3;
+				String field_code = compiler_capture_code_until_php_close(content, i);
+				if(escape_field)
+					parsed_content.append("print(html_escape( " + field_code + " )); ");
+				else
+					parsed_content.append("print( " + field_code + " ); ");
+				continue;
+			}
+
+			if(c == '<' && c1 == '?')
+			{
+				compiler_append_text_literal_output(parsed_content, literal_buffer);
+				inside_literal = false;
+				code_state = CompilerCodeState();
+				i += 1;
+				continue;
+			}
+
+			if(c == '<' && c1 == '/' && c2 == '>')
+			{
+				compiler_append_text_literal_output(parsed_content, literal_buffer);
+				inside_literal = false;
+				i += 2;
+				continue;
+			}
+
+			literal_buffer.append(1, c);
 			continue;
 		}
 
+		if(compiler_code_state_is_neutral(code_state) && ((c == '<' && c1 == '>') || (c == '?' && c1 == '>')))
+		{
+			inside_literal = true;
+			literal_buffer = "";
+			current_line = "";
+			i += 1;
+			continue;
+		}
+
+		current_line.append(1, c);
+
 		compiler_code_state_consume(code_state, parsed_content, content, i);
 
 		if(c == 10 && current_line.substr(0, 6) == "#load ")
@@ -361,6 +432,9 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
 			current_line = "";
 	}
 
+	if(inside_literal)
+		compiler_append_text_literal_output(parsed_content, literal_buffer);
+
 	return(parsed_content);
 }