[OCaml] Mobile-friendly clone of cgit.
perf Lower the LCS matrix cap to 1M cells
Four million int cells put one diff request at roughly 32 MB; concurrent diff views multiplied that. The cap is now one million cells (about 8 MB) with the threshold and fallback shape documented, settling the module TODO.
Changed files
lib/line_diff.ml
@@ -3,16 +3,15 @@
3
3
Computes a longest-common-subsequence diff and groups the result into hunks
4
4
with the surrounding context, which is the shape a reader expects from
5
5
[git diff]. Large inputs fall back to a coarser result rather than spending
6
Removed:
unbounded time and memory on the LCS matrix.
6
Added:
unbounded time and memory on the LCS matrix: when the matrix would exceed
7
Added:
[max_matrix_cells] cells, the diff lists every old line as deleted
8
Added:
followed by every new line as added.
7
9
8
10
The module is named for the granularity it works at: whole lines, matched as
9
11
opaque units, with no word- or character-level refinement.
10
12
11
Removed:
It produces data only. Rendering it is {!module:Ui.Diff}'s job.
13
Added:
It produces data only. Rendering it is {!module:Ui.Diff}'s job. *)
12
14
13
Removed:
TODO: document the exact size threshold and the shape of the fallback, once
14
Removed:
the limit is settled. *)
15
Removed:
16
15
type line_kind = Context | Addition | Deletion
17
16
18
17
type line = {
@@ -49,6 +48,12 @@
49
48
List.rev lines |> List.tl |> List.rev
50
49
else lines
51
50
51
Added:
(* One matrix cell is one word, so the cap bounds a single diff's matrix at
52
Added:
about 8 MB on a 64-bit system. Concurrent requests each allocate their own
53
Added:
matrix, which is why the bound stays modest. Beyond it the diff degrades to
54
Added:
all-deletions-then-all-additions rather than failing. *)
55
Added:
let max_matrix_cells = 1_000_000
56
Added:
52
57
let of_contents old_content new_content =
53
58
let old_lines = Array.of_list (split_lines old_content) in
54
59
let new_lines = Array.of_list (split_lines new_content) in
@@ -79,7 +84,7 @@
79
84
}
80
85
:: acc)
81
86
in
82
Removed:
if matrix_size > 4_000_000 then all_deletions 0 [] @ all_additions 0 []
87
Added:
if matrix_size > max_matrix_cells then all_deletions 0 [] @ all_additions 0 []
83
88
else
84
89
let lengths = Array.make_matrix (old_length + 1) (new_length + 1) 0 in
85
90
for old_index = old_length - 1 downto 0 do
lib/line_diff.mli
@@ -54,7 +54,8 @@
54
54
val of_contents : string -> string -> line list
55
55
(** [of_contents old_content new_content] produces a flat list of diff lines
56
56
between two file contents. Falls back to listing all deletions followed by
57
Removed:
all additions when the LCS matrix would exceed 4 million cells. *)
57
Added:
all additions when the LCS matrix would exceed one million cells (about
58
Added:
8 MB), so one request cannot hold an arbitrarily large matrix. *)
58
59
59
60
val hunks : ?context:int -> line list -> hunk list
60
61
(** Group diff lines into hunks with [context] lines of surrounding unchanged