[_] refactor(look_up): remove look up table references

[apsantiso]

The look_up table takes up a lot of space when you sum up all its fields (UUIDs, plain name, and tokenized name). The look_up table scan is faster since the tokenized field is pre-calculated, however we can accomplish the same result by querying the files and folders tables directly and free up the storage look_up is using.

Plan

1. Remove the look_up table first by removing any references to it, regardless of performance.
2. Optimize the fuzzy search directly on files/folders.

Any optimization applied to files/folders will use less space than the look_up table with all its fields combined.

Performance

New query plan:

Limit  (cost=28823.71..28823.73 rows=10 width=200) (actual time=7072.011..7072.014 rows=10 loops=1)
  Buffers: shared hit=15005 read=18827
  ->  Sort  (cost=28823.71..28849.75 rows=10419 width=200) (actual time=7072.009..7072.011 rows=10 loops=1)
        Sort Key: (CASE WHEN ((f.plain_name)::text = 'xx'::text) THEN 1 ELSE 0 END) DESC, (similarity((f.plain_name)::text, 'xx'::text)) DESC
        Sort Method: top-N heapsort  Memory: 28kB
        Buffers: shared hit=15005 read=18827
        ->  Append  (cost=0.70..28598.56 rows=10419 width=200) (actual time=4.831..7069.302 rows=1163 loops=1)
              Buffers: shared hit=15005 read=18827
              ->  Index Scan using idx_user_status_updated_uuid on files f  (cost=0.70..23754.23 rows=8371 width=192) (actual time=4.830..6861.404 rows=954 loops=1)
                    Index Cond: ((user_id = X) AND (status = 'EXISTS'::enum_files_status))
                    Filter: (((plain_name)::text % 'XX'::text) OR (similarity((plain_name)::text, 'XX'::text) > '0'::double precision))
                    Rows Removed by Filter: 40675
                    Buffers: shared hit=14633 read=18050
              ->  Index Scan using idx_folders_user_parentuuid_plainname_not_deleted_removed on folders fo  (cost=0.57..4792.23 rows=2048 width=234) (actual time=4.483..207.269 rows=209 loops=1)
                    Index Cond: ((user_id = X) AND (parent_uuid IS NOT NULL))
                    Filter: (((plain_name)::text % 'XXX'::text) OR (similarity((plain_name)::text, 'XX'::text) > '0'::double precision))
                    Rows Removed by Filter: 964
                    Buffers: shared hit=372 read=777
Planning:
  Buffers: shared hit=168
Planning Time: 8.127 ms
Execution Time: 7072.138 ms

Old query:

Limit  (cost=78304.95..78360.31 rows=10 width=251) (actual time=626.140..662.270 rows=10 loops=1)
  Buffers: shared hit=225 read=16066
  ->  Nested Loop Left Join  (cost=78304.95..204858.51 rows=22858 width=251) (actual time=626.138..662.262 rows=10 loops=1)
        Buffers: shared hit=225 read=16066
        ->  Nested Loop Left Join  (cost=78304.38..140147.36 rows=22858 width=235) (actual time=625.751..655.564 rows=10 loops=1)
              Buffers: shared hit=204 read=16038
              ->  Gather Merge  (cost=78303.80..80965.99 rows=22858 width=146) (actual time=622.422..627.152 rows=10 loops=1)
                    Workers Planned: 2
                    Workers Launched: 1
                    Buffers: shared hit=183 read=16018
                    ->  Sort  (cost=77303.78..77327.59 rows=9524 width=146) (actual time=615.530..615.543 rows=168 loops=2)
                          Sort Key: (CASE WHEN (("LookUpModel".name)::text = 'giappo'::text) THEN 1 ELSE 0 END) DESC, (NULLIF(ts_rank("LookUpModel".tokenized_name, to_tsquery('giappo'::text)), '1'::double precision)) DESC, (similarity(("LookUpModel".name)::text, 'giappo'::text)) DESC
                          Sort Method: quicksort  Memory: 157kB
                          Buffers: shared hit=183 read=16018
                          Worker 0:  Sort Method: quicksort  Memory: 161kB
                          ->  Parallel Bitmap Heap Scan on look_up "LookUpModel"  (cost=582.48..76674.37 rows=9524 width=146) (actual time=44.631..613.640 rows=595 loops=2)
                                Recheck Cond: ((user_id)::text = '04130834-288f-4c91-938a-179cb31ab91f'::text)
                                Filter: ((to_tsquery('giappo'::text) @@ tokenized_name) OR (similarity((name)::text, 'giappo'::text) > '0'::double precision))
                                Rows Removed by Filter: 20944
                                Heap Blocks: exact=7844
                                Buffers: shared hit=168 read=16018
                                ->  Bitmap Index Scan on user_uuid_look_up_index  (cost=0.00..576.77 rows=68559 width=0) (actual time=14.689..14.689 rows=43079 loops=1)
                                      Index Cond: ((user_id)::text = '04130834-288f-4c91-938a-179cb31ab91f'::text)
                                      Buffers: shared hit=2 read=40
              ->  Index Scan using files_uuid_key on files file  (cost=0.57..2.59 rows=1 width=105) (actual time=2.839..2.839 rows=0 loops=10)
                    Index Cond: (uuid = "LookUpModel".item_id)
                    Buffers: shared hit=21 read=20
        ->  Index Scan using uuid_index on folders folder  (cost=0.57..2.57 rows=1 width=20) (actual time=0.651..0.651 rows=1 loops=10)
              Index Cond: (uuid = "LookUpModel".item_id)
              Buffers: shared hit=21 read=28
Planning:
  Buffers: shared hit=881 read=7
Planning Time: 16.875 ms
Execution Time: 662.877 ms

[sonarqubecloud]

Quality Gate failed

Failed conditions
10.5% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud