feat(html): lay out only the chunks of a sheet near the viewport

docs/design/lazy-loading.md

@@ -191,14 +191,18 @@ is sticky at the top of the wrapper. The row labels stay a `th` in each
 row, and they stay sticky at the left, because each chunk is as wide as
 the sheet.
 
-**Column widths.** All of the tables must have the same columns, so each
-table has `table-layout: fixed`, a `width`, and the same `colgroup`. A
-column that states a width keeps it. Today a column that states no width
-takes the width of its content, for example in a csv. Each chunk table
-would then size it apart. So the writer gives such a column a width: the
-longest text of its first 1000 rows, in `ch`, from 4 to 60. That is how
-LibreOffice finds an optimal width. The cut of a longer text, and the
-spill into the next cells, are the ones a stated width gets today.
+**Column widths.** All of the tables must have the same columns. So each
+table has `table-layout: fixed` and the same `col`s, and states the width
+of the whole sheet (`--odr-sheet-width`, in px). Without that width, a
+table wider than the viewport shrinks each column to what its own cells
+hold, and the chunks do not line up. A column that states a width keeps
+it. A column that states no width, for example in a csv, took the width of
+its content before. Now the writer estimates it from the longest text of
+its first 1000 rows, from 4 to 60 characters, at 7.5px for each character
+and 14px of padding. A cell that wraps its text counts only its longest
+word. The row labels get 7.5px for each digit and 14px. The estimate does
+not know the font of the file, so a wide font can spill a long number past
+its column.
 
 **Where a chunk ends.** A chunk holds at most 256 rows, or as many rows as
 hold 4096 cells, and at least 8 rows. A chunk ends only where no `rowspan`
@@ -233,9 +237,13 @@ the first chunk of a sheet with shapes is always near: the writer states
 | Formulas in the page | read the text of a cell | no change: the DOM of a hidden chunk is still there |
 
 **What stays.** The limits (`spreadsheet_limit`, `spreadsheet_cell_limit`)
-stay as they are in this step. A sheet of 500 K cells takes 7.3 s to its
-first frame today, and about 3 s of that is the scripts. Whether the
-limits can go up is a measurement after this step.
+stay as they are in this step. A sheet of 500 K cells
+(`efficiency-big-1.ods`) took 7.0 s to its first frame, and takes 4.0 s
+with chunks. About 2 s of the rest is `viewport.js`: to find whether the
+browser adjusts the text size, it sets the zoom of the body twice. Each
+set makes the browser compute the style and the layout of the whole
+document again. Whether the limits can go up is a measurement after that
+is fixed.
 
 ## Order of work
 

src/odr/internal/html/document.cpp

@@ -251,6 +251,7 @@ void write_head(const Document &document, const WritingState &state,
   if (document.document_type() == DocumentType::spreadsheet) {
     write_spreadsheet_style(state);
     write_spreadsheet_dark_style(state);
+    write_chunk_style(state);
   }
   // Last, after the sheets whose rules they stand in for.
   if (StyleRegistry *styles = state.styles();

src/odr/internal/html/document_element.cpp

@@ -180,6 +180,20 @@ TableDimensions html::sheet_rendered_extent(const Sheet &sheet,
 
 namespace {
 
+/// What a sheet reads to give a column that states no width one.
+constexpr std::uint32_t estimated_rows = 1000;
+/// About the width of a character of the sheet's font, and the padding of a
+/// cell, which a column stating no width is estimated with.
+constexpr double character_pixels = 7.5;
+constexpr double cell_padding_pixels = 14;
+/// About the width of a digit of a row label, and the padding around one.
+constexpr double digit_pixels = 7.5;
+constexpr double gutter_padding_pixels = 14;
+/// What a row that states no height is estimated at, before it is laid out.
+constexpr double estimated_row_pixels = 20;
+/// A chunk of a sheet holds about as many cells (`lazy-loading.md` decision 6).
+constexpr std::uint32_t chunk_cells = 4096;
+
 /// Whether a reader sees anything. A bookmark marks a place rather than filling
 /// one, and a span or a link is a style around what it holds, so a paragraph
 /// holding only those is still an empty line.
@@ -606,8 +620,74 @@ void html::translate_sheet(const Sheet &sheet, const WritingState &state) {
   const std::optional<HtmlSheetCut> cut =
       state.config().editable ? sheet_cut(sheet, state.config()) : std::nullopt;
 
+  const ElementRange shapes = sheet.shapes();
+  const bool has_shapes = shapes.begin() != shapes.end();
+
+  // `table-layout:fixed` still sizes the table from its content, so an unbroken
+  // line would widen its column; `max-width:0` takes the cell out of that sum.
+  // Every chunk is a table of its own, so every column needs a width: one that
+  // states none takes its longest text, as LibreOffice's optimal width does.
+  std::vector<std::optional<double>> column_pixels(end_column);
+  std::vector<std::string> column_styles(end_column);
+  for (std::uint32_t column_index = 0; column_index < end_column;
+       ++column_index) {
+    const TableColumnStyle table_column_style =
+        sheet.column_style(column_index);
+    column_pixels[column_index] = css_pixels(table_column_style.width);
+    column_styles[column_index] =
+        translate_table_column_style(table_column_style);
+  }
+  for (std::uint32_t column_index = 0; column_index < end_column;
+       ++column_index) {
+    if (column_pixels[column_index].has_value()) {
+      continue;
+    }
+    std::size_t characters = 0;
+    for (std::uint32_t row_index = 0;
+         row_index < std::min(end_row, estimated_rows); ++row_index) {
+      const SheetCell cell = sheet.cell(column_index, row_index);
+      if (cell.is_covered()) {
+        continue;
+      }
+      const std::string text = shown_text(cell);
+      // a cell that wraps its text needs only its longest word
+      const char *breaks =
+          sheet.cell_style(column_index, row_index).wrap_text.value_or(false)
+              ? "\n "
+              : "\n";
+      for (std::size_t begin = 0; begin <= text.size();) {
+        const std::size_t end =
+            std::min(text.find_first_of(breaks, begin), text.size());
+        characters = std::max(characters, util::string::utf8_length(
+                                              text.substr(begin, end - begin)));
+        begin = end + 1;
+      }
+    }
+    const double pixels =
+        static_cast<double>(std::clamp<std::size_t>(characters, 4, 60)) *
+            character_pixels +
+        cell_padding_pixels;
+    column_pixels[column_index] = pixels;
+    column_styles[column_index] +=
+        "width:" + util::number::to_string_significant(pixels, 7) + "px;";
+  }
+
+  const auto write_columns = [&] {
+    state.out().write_element_begin("col",
+                                    HtmlElementOptions()
+                                        .set_close_type(HtmlCloseType::none)
+                                        .set_class("odr-sheet-gutter"));
+    for (std::uint32_t column_index = 0; column_index < end_column;
+         ++column_index) {
+      state.out().write_element_begin(
+          "col", HtmlElementOptions()
+                     .set_close_type(HtmlCloseType::none)
+                     .set_style(column_styles[column_index], state.styles()));
+    }
+  };
+
   state.out().write_element_begin(
-      "table",
+      "div",
       HtmlElementOptions()
           .set_class("odr-sheet")
           .set_attributes([&](const HtmlAttributeWriterCallback &clb) {
@@ -622,56 +702,46 @@ void html::translate_sheet(const Sheet &sheet, const WritingState &state) {
                                       std::to_string(cut->content.rows));
             }
           })
-          .set_style([&]() -> std::optional<HtmlWritable> {
-            if (!print_fit.has_value()) {
-              return std::nullopt;
+          .set_style([&]() -> std::string {
+            // Every table states the same width, so `table-layout:fixed`
+            // takes the columns from the `col`s alone. A table wider than the
+            // viewport would else shrink a column to what its cells hold.
+            const double gutter =
+                static_cast<double>(
+                    TablePosition::to_row_string(end_row == 0 ? 0 : end_row - 1)
+                        .size()) *
+                    digit_pixels +
+                gutter_padding_pixels;
+            double width = gutter;
+            for (const std::optional<double> &pixels : column_pixels) {
+              width += pixels.value_or(0);
+            }
+            std::string style = "--odr-sheet-gutter:" +
+                                util::number::to_string_significant(gutter, 7) +
+                                "px;--odr-sheet-width:" +
+                                util::number::to_string_significant(width, 7) +
+                                "px;";
+            if (print_fit.has_value()) {
+              // `Measure` renders no exponent form
+              style += "--odr-print-fit:" +
+                       Measure(*print_fit, DynamicUnit()).to_string() + ";";
             }
-            // `Measure` renders no exponent form
-            return "--odr-print-fit:" +
-                   Measure(*print_fit, DynamicUnit()).to_string() + ";";
+            return style;
           }()));
 
-  state.out().write_element_begin("col",
-                                  HtmlElementOptions()
-                                      .set_close_type(HtmlCloseType::none)
-                                      .set_class("odr-sheet-gutter"));
-
-  // `table-layout:fixed` still sizes the table from its content, so an unbroken
-  // line would widen its column; `max-width:0` takes the cell out of that sum.
-  // Only where a width is stated: a column that states none is its content's.
-  std::vector<std::optional<double>> column_pixels(end_column);
-
-  for (std::uint32_t column_index = 0; column_index < end_column;
-       ++column_index) {
-    const TableColumnStyle table_column_style =
-        sheet.column_style(column_index);
-    column_pixels[column_index] = css_pixels(table_column_style.width);
-
-    state.out().write_element_begin(
-        "col", HtmlElementOptions()
-                   .set_close_type(HtmlCloseType::none)
-                   .set_style(translate_table_column_style(table_column_style),
-                              state.styles()));
-  }
-
   // No `scope`: the letters and numbers are a ruler, not headers of what they
-  // label.
+  // label. A table of its own, because a sticky `thead` sticks only inside
+  // its table, and the rows are in many.
   {
+    state.out().write_element_begin(
+        "table", HtmlElementOptions().set_class("odr-sheet-head"));
+    write_columns();
     state.out().write_element_begin("thead");
     state.out().write_element_begin("tr");
 
-    // Under `table-layout:fixed` the first row sizes the columns, and `ch`
-    // resolves against the ruler's font — so the gutter width sits here rather
-    // than on the `<col>`.
     state.out().write_element_begin(
         "th",
-        HtmlElementOptions()
-            .set_inline(true)
-            .set_class("odr-sheet-corner")
-            .set_style("width:calc(" +
-                       std::to_string(
-                           TablePosition::to_row_string(end_row - 1).size()) +
-                       "ch + 14px);"));
+        HtmlElementOptions().set_inline(true).set_class("odr-sheet-corner"));
     state.out().write_element_end("th");
 
     for (std::uint32_t column_index = 0; column_index < end_column;
@@ -685,19 +755,55 @@ void html::translate_sheet(const Sheet &sheet, const WritingState &state) {
 
     state.out().write_element_end("tr");
     state.out().write_element_end("thead");
+    state.out().write_element_end("table");
   }
 
-  state.out().write_element_begin("tbody");
-
-  const ElementRange shapes = sheet.shapes();
-  const bool has_shapes = shapes.begin() != shapes.end();
+  // a chunk ends where no merge is open, so no `rowspan` crosses two tables
+  const std::uint32_t chunk_rows = std::clamp<std::uint32_t>(
+      chunk_cells / std::max<std::uint32_t>(end_column, 1), 8, 256);
+  std::uint32_t rows_in_chunk = 0;
+  bool chunk_open = false;
+  const auto begin_chunk = [&](const std::uint32_t first_row) {
+    double height = 0;
+    for (std::uint32_t row_index = first_row;
+         row_index < std::min(end_row, first_row + chunk_rows); ++row_index) {
+      height += css_pixels(sheet.row_style(row_index).height)
+                    .value_or(estimated_row_pixels);
+    }
+    state.out().write_element_begin(
+        "table",
+        HtmlElementOptions()
+            .set_class(first_row == 0 ? "odr-sheet-rows odr-chunk "
+                                        "odr-chunk-near"
+                                      : "odr-sheet-rows odr-chunk")
+            .set_attributes([&](const HtmlAttributeWriterCallback &clb) {
+              // a shape hangs from A1 and may reach past the first chunk
+              if (first_row == 0 && has_shapes) {
+                clb("data-odr-chunk-held", "");
+              }
+            })
+            .set_style("contain-intrinsic-height:auto " +
+                       util::number::to_string_significant(height, 7) + "px"));
+    write_columns();
+    state.out().write_element_begin("tbody");
+    chunk_open = true;
+    rows_in_chunk = 0;
+  };
+  const auto end_chunk = [&] {
+    state.out().write_element_end("tbody");
+    state.out().write_element_end("table");
+    chunk_open = false;
+  };
 
   // per column, the row where a cell spanning down from above ends
   std::vector<std::uint32_t> span_end_row(end_column, 0);
 
   TableCursor cursor;
   for (std::uint32_t row_index = cursor.row(); row_index < end_row;
        row_index = cursor.row()) {
+    if (!chunk_open) {
+      begin_chunk(row_index);
+    }
     const TableRowStyle table_row_style = sheet.row_style(row_index);
 
     state.out().write_element_begin(
@@ -932,10 +1038,18 @@ void html::translate_sheet(const Sheet &sheet, const WritingState &state) {
     state.out().write_element_end("tr");
 
     cursor.add_row();
+    if (++rows_in_chunk >= chunk_rows &&
+        std::ranges::none_of(span_end_row, [&](const std::uint32_t span_end) {
+          return span_end > cursor.row();
+        })) {
+      end_chunk();
+    }
+  }
+  if (chunk_open) {
+    end_chunk();
   }
 
-  state.out().write_element_end("tbody");
-  state.out().write_element_end("table");
+  state.out().write_element_end("div");
 }
 
 namespace {

src/odr/internal/html/frontend/search.js

@@ -161,7 +161,7 @@
       mark.classList.add("current");
     });
     // A hit inside a folded section, or in a chunk the browser does not lay
-    // out, is scrolled to but not shown.
+    // out, is shown before it is scrolled to.
     for (
       var element = marks[current][0].parentElement;
       element !== null;
@@ -170,9 +170,9 @@
       if (element.nodeName === "DETAILS") {
         element.open = true;
       }
-      if (element.classList.contains("odr-chunk")) {
-        element.classList.add("odr-chunk-near");
-      }
+    }
+    if (odr.chunks !== undefined) {
+      odr.chunks.show(marks[current][0]);
     }
     marks[current][0].scrollIntoView({ block: "center", inline: "center" });
   }

src/odr/internal/html/frontend/sheet-editing.js

@@ -1535,6 +1535,10 @@
   // Offsets, not rects: blink scales a rect by the body zoom `viewport_js`
   // applies, and the overlay is laid out under that zoom.
   function place(cell) {
+    // a cell in a far chunk has no geometry until the chunk is laid out
+    if (odr.chunks !== undefined) {
+      odr.chunks.show(cell);
+    }
     var left = 0;
     var top = 0;
     for (var node = cell; node !== null; node = node.offsetParent) {

src/odr/internal/html/frontend/spreadsheet-dark.css

@@ -13,4 +13,4 @@
 --odr-sheet-stale:#e3b341;
 --odr-sheet-raised:#1c2128;
 }
-.odr-sheet{background-color:#161b22!important}
+.odr-sheet-head,.odr-sheet-rows{background-color:#161b22!important}

src/odr/internal/html/frontend/spreadsheet.css

@@ -16,14 +16,21 @@
 }
 /* A sheet is not a page: past the last row and column is canvas. */
 body{margin:0;background:var(--odr-sheet-canvas)}
-.odr-sheet{background:#fff;border-collapse:collapse;table-layout:fixed}
+/* A sheet is the ruler and a table for each chunk of rows (`lazy-loading.md`
+   decision 6). Every column states its width, so the tables line up. The
+   ruler sticks to the top of the whole sheet, not of one table. */
+.odr-sheet-head,.odr-sheet-rows{background:#fff;border-collapse:collapse;table-layout:fixed;width:var(--odr-sheet-width)}
+.odr-sheet-head{position:sticky;top:0;z-index:2}
+/* The border between two chunks is drawn once, as inside a table. */
+.odr-sheet-rows+.odr-sheet-rows{margin-top:-1px}
+.odr-sheet col.odr-sheet-gutter{width:var(--odr-sheet-gutter)}
 /* The sheet's own cells, not a table the document itself drew inside one. The
    font is what anything in a cell falls back to, a string written straight
    into it included. */
-.odr-sheet>tbody>tr>td{vertical-align:bottom;height:inherit;padding:1px 6px;font-family:var(--odr-sheet-font);font-size:10pt}
+.odr-sheet-rows>tbody>tr>td{vertical-align:bottom;height:inherit;padding:1px 6px;font-family:var(--odr-sheet-font);font-size:10pt}
 /* Exactly the cell's height, which is what holds a row to the height the file
    states. `translate_sheet` says per cell what a line too long for it does. */
-.odr-sheet>tbody>tr>td>x-p{height:inherit}
+.odr-sheet-rows>tbody>tr>td>x-p{height:inherit}
 /* Sticky cells in a collapsed border model do not repaint their borders in
    Chrome or WebKit, so the ruler uses inset shadows. */
 .odr-sheet th{position:sticky;background:var(--odr-sheet-ruler);color:var(--odr-sheet-ruler-text);font:500 12px/1.6 var(--odr-sheet-font);text-align:center;vertical-align:middle;padding:0 4px;white-space:nowrap;user-select:none}
@@ -72,8 +79,9 @@ body{margin:0;background:var(--odr-sheet-canvas)}
    width. The ruler collapses rather than `display:none`, which would take the
    cells out of their rows and shift every column left. */
 @media print{
-.odr-sheet thead{display:none}
-.odr-sheet-gutter{visibility:collapse;width:0}
+.odr-sheet-head{display:none}
+.odr-sheet col.odr-sheet-gutter{visibility:collapse;width:0}
+.odr-sheet-head,.odr-sheet-rows{width:calc(var(--odr-sheet-width) - var(--odr-sheet-gutter))}
 .odr-sheet-row-header{visibility:collapse;padding:0;box-shadow:none}
 .odr-sheet{zoom:var(--odr-print-fit,1);max-width:100%}
 .odr-sheet col{min-width:0!important}