From c45dade4478bc92180323cc1555300cb6e467b05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Fri, 27 Jan 2023 18:43:48 +0300 Subject: [PATCH 01/22] Refactor README.md By https://github.com/ibarwick/firebird_fdw/blob/master/README.md example of advanced FDW documentation. Add about functions, encodings, identifiers, FDW options and types of options, refactor features, notes and limitations. --- README.md | 473 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 377 insertions(+), 96 deletions(-) diff --git a/README.md b/README.md index 8f6f31ae..696cdbdf 100644 --- a/README.md +++ b/README.md @@ -1,133 +1,414 @@ -# SQLite Foreign Data Wrapper for PostgreSQL -This PostgreSQL extension is a Foreign Data Wrapper for [SQLite][1]. +SQLite Foreign Data Wrapper for PostgreSQL +========================================== -The current version can work with PostgreSQL 11, 12, 13, 14 and 15. +This is a foreign data wrapper (FDW) to connect [PostgreSQL](https://www.postgresql.org/) 11+ +to [SQLite](https://sqlite.org/) database file. -## Installation -### 1. Install SQLite library +Contents +-------- -For debian or ubuntu: -
-apt-get install libsqlite3-dev
-
+1. [Features](#features) +2. [Supported platforms](#supported-platforms) +3. [Installation](#installation) +4. [Usage](#usage) +5. [Functions](#functions) +6. [Identifier case handling](#identifier-case-handling) +7. [Generated columns](#generated-columns) +8. [Character set handling](#character-set-handling) +9. [Examples](#examples) +10. [Limitations](#limitations) +13. [Useful links](#useful-links) -You can also [download SQLite source code][2] and [build SQLite][3]. +Features +-------- -### 2. Build and install sqlite_fdw +### Common features +- Transactions +- Support `INSERT`/`UPDATE`/`DELETE` (both Direct modification and Foreign modification). +- Support `TRUNCATE` by deparsing into `DELETE` statement without `WHERE` clause +- Allow control over whether foreign servers keep connections open after transaction completion. This is controlled by `keep_connections` and defaults to on +- Support list cached connections to foreign servers by using function `sqlite_fdw_get_connections()` +- Support discard cached connections to foreign servers by using function `sqlite_fdw_disconnect()`, `sqlite_fdw_disconnect_all()`. +- Support Bulk `INSERT` by using `batch_size` option +- Support `INSERT`/`UPDATE` with generated column +- Support `ON CONFLICT DO NOTHING`. -Add a directory of pg_config to PATH and build and install sqlite_fdw. -
+### Pushdowning
+- `WHERE` clauses are pushdowned
+- Aggregate function are pushdowned
+- `ORDER BY` is pushdowned
+- Joins (left/right/inner/cross) are pushdowned
+- `CASE` expressions are pushdowned.
+- `LIMIT` and `OFFSET` are pushdowned (*when all tables queried are fdw)
+- Support `GROUP BY`, `HAVING` push-down.
+- `upper`, `lower` and other character case functions are **not** pushed down because they does not work with UNICODE character in SQLite.
+- `WITH TIES` option is **not** pushed down.
+
+### Notes about pushdowning
+
+- For push-down case, the number after floating point may be different from the result of PostgreSQL.
+
+### Notes about features
+- SQLite evaluates division by zero as `NULL`. It is different from PostgreSQL, which will display `Division by zero` error.
+- The data type of column of foreign table should match with data type of column in SQLite to avoid wrong result. For example, if the column of SQLite is `float` (which will be stored as `float8`), the column of foreign table should be `float8`, too. If the column of foreign table is `float4`, it may cause wrong result when select.
+- For `key` option, user needs to specify the primary key column of SQLite table corresponding with the `key` option. If not, wrong result may occur when update or delete.
+- When `Sum` of data in table is out of range, `sqlite_fdw` will display `Infinity` value. It is different from PostgreSQL FDW, which will display `ERROR: value out of range: overflow` error.
+- For `numeric` data type, `sqlite_fdw` use `sqlite3_column_double` to get value, while SQLite shell uses `sqlite3_column_text` to get value. Those 2 APIs may return different numeric value. Therefore, for `numeric` data type, the value returned from `sqlite_fdw` may different from the value returned from SQLite shell.
+- `sqlite_fdw` can return implementation-dependent order for column if the column is not specified in `ORDER BY` clause.
+- When the column type is `varchar array`, if the string is shorter than the declared length, values of type character will be space-padded; values of type `character varying` will simply store the shorter string.
+
+Also see [Limitations](#limitations)
+
+Supported platforms
+-------------------
+
+`sqlite_fdw` was developed on Linux and should run on any
+reasonably POSIX-compliant system.
+
+Installation
+------------
+
+For some Linux distributives deb and rpm packages are avalillable.
+
+### Source installation
+
+Prerequisites:
+* `libsqlite3-dev`, especially `sqlite.h`
+* `gcc`
+* `make`
+
+#### 1. Install SQLite library
+
+For Debian or Ubuntu:
+`apt-get install libsqlite3-dev`
+
+You can also [download SQLite source code][1] and [build SQLite][2].
+
+#### 2. Build and install sqlite_fdw
+
+Add a directory of `pg_config` to PATH and build and install `sqlite_fdw`.
+
+```sh
 make USE_PGXS=1
 make install USE_PGXS=1
-
+``` -If you want to build sqlite_fdw in a source tree of PostgreSQL, use -
+If you want to build `sqlite_fdw` in a source tree of PostgreSQL, use
+```sh
 make
 make install
-
+``` + +Usage +----- + +## CREATE SERVER options + +`sqlite_fdw` accepts the following options via the `CREATE SERVER` command: + +- **database** as *string*, **required** + + SQLite database path. + +- **truncatable** as *boolean*, optional + + Allows foreign tables to be truncated using the `TRUNCATE` command. + +- **keep_connections** as *boolean*, optional + + Allows to keep connections to SQLite while there is no SQL operations between PostgreSQL and SQLite. + +- **batch_size** as *integer*, optional + + Specifies the number of rows which should be inserted in a single `INSERT` operation. This setting can be overridden for individual tables. + + +## CREATE USER MAPPING options + +`sqlite_fdw` no need any `CREATE USER MAPPING` command. + +Just ensure `postgres` OS user have permissions for reading or writing on SQLite database file. + + +## CREATE FOREIGN TABLE options + +`sqlite_fdw` accepts the following table-level options via the +`CREATE FOREIGN TABLE` command: + +- **table** as *string*, **required** + + SQLite table name. + +- **truncatable** as *boolean*, optional + + Allows table to be truncated using the `TRUNCATE` command. + +- **batch_size** as *integer*, optional + + See `CREATE SERVER` options section for details. + +`sqlite_fdw` accepts the following column-level options via the +`CREATE FOREIGN TABLE` command: + +- **column_name** as *string*, optional + + This option gives the column name to use for the column on the remote server. + +- **column_type** as *string*, optional + + Option to convert INT SQLite column (epoch Unix Time) to be treated/visualized as TIMESTAMP in PostgreSQL. + +- **key** as *boolean*, optional + + Indicates a column as a part of primary key or unique key of SQLite table. + +## IMPORT FOREIGN SCHEMA options + +`sqlite_fdw` supports [IMPORT FOREIGN SCHEMA](https://www.postgresql.org/docs/current/sql-importforeignschema.html) +(PostgreSQL 9.5+) and accepts no custom options for this command. + +## TRUNCATE support + +`sqlite_fdw` implements the foreign data wrapper `TRUNCATE` API, available +from PostgreSQL 14. + +As SQlite does not provide a `TRUNCATE` command, it is simulated with a +simple unqualified `DELETE` operation. + +Following restrictions apply: -## Usage + - `TRUNCATE ... CASCADE` is not supported + - `TRUNCATE ... RESTART IDENTITY` is not supported + - SQLlite tables with foreign key references can cause errors during truncating -### FDW options +Functions +--------- -| **No** | Option name | Context | Required | Description | -|--------|-------------|---------|----------|-------------| -| 1 | database | SERVER | Required | SQLite database path. | -| 2 | table | FOREIGN TABLE | Required | SQLite table name. | -| 3 | key | ATTRIBUTE | Optional | Primary key or unique key of SQLite table. | -| 4 | column_type | ATTRIBUTE | Optional | Option to convert INT SQLite column (epoch Unix Time) to be treated/visualized as TIMESTAMP in PostgreSQL. | -| 5 | column_name | ATTRIBUTE | Optional | This option gives the column name to use for the column on the remote server. | -| 6 | truncatable | SERVER,
FOREIGN TABLE | Optional | This option controls whether sqlite_fdw allows foreign tables to be truncated using the TRUNCATE command. | +As well as the standard `sqlite_fdw_handler()` and `sqlite_fdw_validator()` +functions, `sqlite_fdw` provides the following user-callable utility functions: -### Load extension -
-CREATE EXTENSION sqlite_fdw;
-
+- SETOF record **sqlite_fdw_get_connections(server_name text, valid bool)** -### Create server -Please specify SQLite database path using `database` option: -
-CREATE SERVER sqlite_server FOREIGN DATA WRAPPER sqlite_fdw OPTIONS (database '/tmp/test.db');
-
+- bool **sqlite_fdw_disconnect(text)** + Closes connection from PostgreSQL to SQLite in the current session. + +- bool **sqlite_fdw_disconnect_all()** + +- SETOF record **sqlite_fdw_version()**; + +Identifier case handling +------------------------ + +PostgreSQL folds identifiers to lower case by default, SQlite is case insensetive by default. It's important +to be aware of potential issues with table and column names. + +This SQL isn't correct for SQLite: `Error: duplicate column name: a`, but is correct for PostgreSQL + +```sql + CREATE TABLE T ( + "A" INTEGER, + "a" NUMERIC + ); +``` +For SQLite there is no difference between + +```sql + select * from t; + select * from T; + select * from "t"; + select * from "T"; +``` +For PostgreSQL `select * from "T";` is independend query to table `T`, not to table `t` as first queries. + +If there is + +```sql + CREATE TABLE T ( + A INTEGER, + b REAL + ); +``` +in SQLite, both `a` and `A` , `b` and `B` columns will have the same real datasource in SQlite + +```sql + CREATE FOREIGN TABLE "SQLite test" ( + "A" int4 NULL, + "B" float8 NULL, + "a" int8 NULL, + "b" numeric NULL + ) + SERVER sqlite_server + OPTIONS (table 'T'); +``` + +Generated columns +----------------- + +SQLite provides support for [generated columns](https://www.sqlite.org/gencol.html). +Behaviour of `sqlite_fdw` with this columns _isn't yet described_. + +Note that while `sqlite_fdw` will `insert` or `update` the generated column value +in SQLite, there is nothing to stop the value being modified within SQLite, +and hence no guarantee that in subsequent `SELECT` operations the column will +still contain the expected generated value. This limitation also applies to +`postgres_fdw`. + +For more details on generated columns see: + +- [Generated Columns](https://www.postgresql.org/docs/current/ddl-generated-columns.html) +- [CREATE FOREIGN TABLE](https://www.postgresql.org/docs/current/sql-createforeigntable.html) + +Character set handling +---------------------- + +When `sqlite_fdw` connects to a SQLite [no character set metadata](https://www.sqlite.org/search?s=d&q=character+set) +stored in SQLite. There is only [`PRAGMA encoding;`](https://www.sqlite.org/pragma.html#pragma_encoding) with UTF-only values (`UTF-8`, `UTF-16`, `UTF-16le`, `UTF-16be`). All strings are interpreted acording the PostgreSQL database's server encoding. It's not a problem +if both PostgreSQL database and SQLite character data from database file has UTF-8 or UTF-16 encoding. Otherewise +character interpretation transformation problems will occur. + +Character case functions such as `upper`, `lower` and other are not pushed down because they does not work with UNICODE character in SQLite. + +Examples +-------- + +### Install the extension: + +```sql + CREATE EXTENSION sqlite_fdw; +``` + +### Create a foreign server with appropriate configuration: + +Please specify SQLite database path using `database` option. + +```sql + CREATE SERVER sqlite_server + FOREIGN DATA WRAPPER sqlite_fdw + OPTIONS ( + database '/path/to/database' + ); +``` + +### No need user mapping ### Create foreign table Please specify `table` option if SQLite table name is different from foreign table name. -
-CREATE FOREIGN TABLE t1(a integer, b text) SERVER sqlite_server OPTIONS (table 't1_sqlite');
-
+ +```sql + CREATE FOREIGN TABLE t1 ( + a integer, + b text) + SERVER sqlite_server + OPTIONS ( + table 't1_sqlite' + ); +``` If you want to update tables, please add `OPTIONS (key 'true')` to a primary key or unique key like the following: -
-CREATE FOREIGN TABLE t1(a integer OPTIONS (key 'true'), b text) SERVER sqlite_server OPTIONS (table 't1_sqlite');
-
- -If you need to convert INT SQLite column (epoch Unix Time) to be treated/visualized as TIMESTAMP in PostgreSQL, please add `OPTIONS (column_type 'INT')` when -defining FOREIGN table at PostgreSQL like the following: -
-CREATE FOREIGN TABLE t1(a integer, b text, c timestamp without time zone OPTIONS (column_type 'INT')) SERVER sqlite_server OPTIONS (table 't1_sqlite');
-
-### Import foreign schema -
-IMPORT FOREIGN SCHEMA public FROM SERVER sqlite_server INTO public;
-
+ +```sql + CREATE FOREIGN TABLE t1( + a integer OPTIONS (key 'true'), + b text) + SERVER sqlite_server + OPTIONS ( + table 't1_sqlite' + ); +``` + +If you need to convert INT SQLite column (epoch Unix Time) to be treated/visualized as `TIMESTAMP` in PostgreSQL, please add `OPTIONS (column_type 'INT')` when defining FOREIGN table at PostgreSQL like the following: + +```sql + CREATE FOREIGN TABLE t1( + a integer, + b text, + c timestamp without time zone OPTIONS (column_type 'INT')) + SERVER sqlite_server + OPTIONS ( + table 't1_sqlite' + ); +``` + +As above, but with aliased column names: + +```sql + CREATE FOREIGN TABLE t1( + a integer, + b text OPTIONS (column_name 'test_id'), + c timestamp without time zone OPTIONS (column_type 'INT', column_name 'unixtime') + SERVER sqlite_server + OPTIONS ( + table 't1_sqlite' + ); +``` + +### Import a SQLite database as schema to PostgreSQL: + +```sql + IMPORT FOREIGN SCHEMA someschema + FROM SERVER sqlite_server; + INTO public; +``` + +Note: `someschema` has no particular meaning and can be set to an arbitrary value. ### Access foreign table -
+For the table from previous examples
+
+```sql
 SELECT * FROM t1;
-
+``` + +Limitations +----------- -## Features -- Support INSERT/UPDATE/DELETE (both Direct modification and Foreign modification). -- WHERE clauses are pushdowned -- Aggregate function are pushdowned -- Order By is pushdowned -- Joins (left/right/inner/cross) are pushdowned -- CASE expressions are pushdowned. -- Limit and Offset are pushdowned (*when all tables queried are fdw) -- Transactions -- Support TRUNCATE by deparsing into DELETE statement without WHERE clause -- Allow control over whether foreign servers keep connections open after transaction completion. This is controlled by `keep_connections` and defaults to on -- Support list cached connections to foreign servers by using function sqlite_fdw_get_connections() -- Support discard cached connections to foreign servers by using function sqlite_fdw_disconnect(), sqlite_fdw_disconnect_all(). -- Support Bulk Insert by using batch_size option -- Support Insert/Update with generated column -- Support GROUP BY, HAVING push-down. -- Support ON CONFLICT DO NOTHING. -## Limitations - `COPY` command for foreign tables is not supported -- IMPORT of generated column is not supported +- `IMPORT` of generated column is not supported - Insert into a partitioned table which has foreign partitions is not supported. Error "Not support partition insert" will display. -- TRUNCATE in sqlite_fdw always delete data of both parent and child tables (no matter user inputs `TRUNCATE table CASCADE` or `TRUNCATE table RESTRICT`) if there are foreign-keys references with "ON DELETE CASCADE" clause. -- RETURNING is not supported. - -## Notes -- SQLite evaluates division by zero as NULL. It is different from PostgreSQL, which will display "Division by zero" error. -- The data type of column of foreign table should match with data type of column in SQLite to avoid wrong result. For example, if the column of SQLite is float (which will be stored as float8), the column of foreign table should be float8, too. If the column of foreign table is float4, it may cause wrong result when select. -- For 'key' option, user needs to specify the primary key column of SQLite table corresponding with the 'key' option. If not, wrong result may occur when update or delete. -- When Sum of data in table is out of range, SQLite FDW will display "Infinity" value. It is different from PostgreSQL FDW, which will display "ERROR: value out of range: overflow" error. -- For push-down case, the number after floating point may be different from the result of PostgreSQL. -- For numeric type, SQLite FDW use sqlite3_column_double to get value, while SQLite shell uses sqlite3_column_text to get value. Those 2 APIs may return different numeric value. Therefore, for numeric type, the value returned from SQLite FDW may different from the value returned from SQLite shell. -- SQLite FDW can return implementation-dependent order for column if the column is not specified in ORDER BY clause. -- WITH TIES option is not pushed down. -- upper, lower functions are not pushed down because they does not work with UNICODE character in SQLite. -- When the column type is varchar array, if the string is shorter than the declared length, values of type character will be space-padded; values of type character varying will simply store the shorter string. -- SQLite FDW only supports ARRAY const, for example, ANY (ARRAY[1, 2, 3]) or ANY ('{1, 2 ,3}'). SQlite FDW does not support ARRAY expression, for example, ANY (ARRAY[c1, 1, c1+0]). For ANY(ARRAY) clause, SQLite FDW deparses it using IN operator. -- For sum function of SQLite, output of sum(bigint) is integer value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of bigint, so overflow does not occur. +- `TRUNCATE` in `sqlite_fdw` always delete data of both parent and child tables (no matter user inputs `TRUNCATE table CASCADE` or `TRUNCATE table RESTRICT`) if there are foreign-keys references with `ON DELETE CASCADE` clause. +- `RETURNING` is not supported. +- `sqlite_fdw` only supports `ARRAY` const, for example, `ANY (ARRAY[1, 2, 3])` or `ANY ('{1, 2 ,3}')`. `Sqlite_fdw` does not support `ARRAY` expression, for example, `ANY (ARRAY[c1, 1, c1+0])`. For `ANY(ARRAY)` clause, `sqlite_fdw` deparses it using `IN` operator. +- For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. -## Contributing + +Contributing +------------ + Opening issues and pull requests on GitHub are welcome. -## License +Useful links +------------ + +### Source + + - https://github.com/pgspider/sqlite_fdw + - https://pgxn.org/dist/sqlite_fdw/ + +### General FDW Documentation + + - https://www.postgresql.org/docs/current/ddl-foreign-data.html + - https://www.postgresql.org/docs/current/sql-createforeigndatawrapper.html + - https://www.postgresql.org/docs/current/sql-createforeigntable.html + - https://www.postgresql.org/docs/current/sql-importforeignschema.html + - https://www.postgresql.org/docs/current/fdwhandler.html + - https://www.postgresql.org/docs/current/postgres-fdw.html + +### Other FDWs + + - https://wiki.postgresql.org/wiki/Fdw + - https://pgxn.org/tag/fdw/ + +License +------- + Copyright (c) 2018, TOSHIBA CORPORATION Copyright (c) 2011 - 2016, EnterpriseDB Corporation Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies. -See the [`LICENSE`][4] file for full details. +See the [`License`][3] file for full details. -[1]: https://www.sqlite.org/index.html -[2]: https://www.sqlite.org/download.html -[3]: https://www.sqlite.org/howtocompile.html -[4]: LICENSE +[1]: https://www.sqlite.org/download.html +[2]: https://www.sqlite.org/howtocompile.html +[3]: License From f4d6fb495f9628dddded67b07674c328e9af70bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 09:20:14 +0300 Subject: [PATCH 02/22] Update README.md by PR-review Thanks to https://github.com/t-kataym (author of review) https://github.com/pgspider/sqlite_fdw/pull/54#pullrequestreview-1274494212 --- README.md | 50 ++++++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index 696cdbdf..8605b865 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ SQLite Foreign Data Wrapper for PostgreSQL ========================================== This is a foreign data wrapper (FDW) to connect [PostgreSQL](https://www.postgresql.org/) 11+ -to [SQLite](https://sqlite.org/) database file. +to [SQLite](https://sqlite.org/) database file. This FDW works with PostgreSQL 11, 12, 13, 14 and 15. Contents -------- @@ -17,7 +17,7 @@ Contents 8. [Character set handling](#character-set-handling) 9. [Examples](#examples) 10. [Limitations](#limitations) -13. [Useful links](#useful-links) +11. [Useful links](#useful-links) Features -------- @@ -207,36 +207,36 @@ This SQL isn't correct for SQLite: `Error: duplicate column name: a`, but is cor ```sql CREATE TABLE T ( - "A" INTEGER, - "a" NUMERIC + "A" INTEGER, + "a" NUMERIC ); ``` For SQLite there is no difference between ```sql - select * from t; - select * from T; - select * from "t"; - select * from "T"; + SELECT * FROM t; + SELECT * FROM T; + SELECT * FROM "t"; + SELECT * FROM "T"; ``` -For PostgreSQL `select * from "T";` is independend query to table `T`, not to table `t` as first queries. +For PostgreSQL `SELECT * FROM "T";` is independend query to table `T`, not to table `t` as first queries. If there is ```sql CREATE TABLE T ( - A INTEGER, - b REAL + A INTEGER, + b REAL ); ``` in SQLite, both `a` and `A` , `b` and `B` columns will have the same real datasource in SQlite ```sql CREATE FOREIGN TABLE "SQLite test" ( - "A" int4 NULL, - "B" float8 NULL, - "a" int8 NULL, - "b" numeric NULL + "A" int4 NULL, + "B" float8 NULL, + "a" int8 NULL, + "b" numeric NULL ) SERVER sqlite_server OPTIONS (table 'T'); @@ -283,11 +283,11 @@ Examples Please specify SQLite database path using `database` option. ```sql - CREATE SERVER sqlite_server - FOREIGN DATA WRAPPER sqlite_fdw - OPTIONS ( - database '/path/to/database' - ); + CREATE SERVER sqlite_server + FOREIGN DATA WRAPPER sqlite_fdw + OPTIONS ( + database '/path/to/database' + ); ``` ### No need user mapping @@ -346,9 +346,9 @@ As above, but with aliased column names: ### Import a SQLite database as schema to PostgreSQL: ```sql - IMPORT FOREIGN SCHEMA someschema - FROM SERVER sqlite_server; - INTO public; + IMPORT FOREIGN SCHEMA someschema + FROM SERVER sqlite_server; + INTO public; ``` Note: `someschema` has no particular meaning and can be set to an arbitrary value. @@ -357,7 +357,8 @@ Note: `someschema` has no particular meaning and can be set to an arbitrary valu For the table from previous examples ```sql -SELECT * FROM t1; + SELECT * + FROM t1; ``` Limitations @@ -371,6 +372,7 @@ Limitations - `sqlite_fdw` only supports `ARRAY` const, for example, `ANY (ARRAY[1, 2, 3])` or `ANY ('{1, 2 ,3}')`. `Sqlite_fdw` does not support `ARRAY` expression, for example, `ANY (ARRAY[c1, 1, c1+0])`. For `ANY(ARRAY)` clause, `sqlite_fdw` deparses it using `IN` operator. - For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. +- SQLite does not support `Numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. Contributing ------------ From a8b305051d3214fc80e3fdca093c4e0ca5fe0c26 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 09:21:05 +0300 Subject: [PATCH 03/22] README.md pg versions fix --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8605b865..aad2e585 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ SQLite Foreign Data Wrapper for PostgreSQL ========================================== -This is a foreign data wrapper (FDW) to connect [PostgreSQL](https://www.postgresql.org/) 11+ +This is a foreign data wrapper (FDW) to connect [PostgreSQL](https://www.postgresql.org/) to [SQLite](https://sqlite.org/) database file. This FDW works with PostgreSQL 11, 12, 13, 14 and 15. Contents From 3d24ec996e92633e3ec986cb517121bc9dfb7ba7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 10:00:03 +0300 Subject: [PATCH 04/22] REDME.md fix indent --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index aad2e585..dff57486 100644 --- a/README.md +++ b/README.md @@ -299,7 +299,7 @@ Please specify `table` option if SQLite table name is different from foreign tab CREATE FOREIGN TABLE t1 ( a integer, b text) - SERVER sqlite_server + SERVER sqlite_server OPTIONS ( table 't1_sqlite' ); From 39ba546e9728dcf6c93404b5aea380befbf6941b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 10:13:10 +0300 Subject: [PATCH 05/22] README.md fix SQL indent --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index dff57486..4e4435ce 100644 --- a/README.md +++ b/README.md @@ -275,7 +275,7 @@ Examples ### Install the extension: ```sql - CREATE EXTENSION sqlite_fdw; + CREATE EXTENSION sqlite_fdw; ``` ### Create a foreign server with appropriate configuration: From 25c24c69c87f8cea0cd4f3bf6131eb7fd71558c9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 10:25:08 +0300 Subject: [PATCH 06/22] Add about tests --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 4e4435ce..884eb0b1 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,9 @@ Contents 8. [Character set handling](#character-set-handling) 9. [Examples](#examples) 10. [Limitations](#limitations) -11. [Useful links](#useful-links) +11. [Tests](#tests) +12. [Contributing](#contributing) +13. [Useful links](#useful-links) Features -------- @@ -374,6 +376,10 @@ Limitations - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. - SQLite does not support `Numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. +Tests +----- +*Yet not described*. Some tests as `psql` expected outputs can be found in [expected](expected) directory. + Contributing ------------ From 1dc10cc3c6ce3952db97ea8dafa3de671bded57d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 10:44:41 +0300 Subject: [PATCH 07/22] Fix README.md by review Thanks to https://github.com/son-phamngoc --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 884eb0b1..1a9b5a85 100644 --- a/README.md +++ b/README.md @@ -137,9 +137,9 @@ Just ensure `postgres` OS user have permissions for reading or writing on SQLite `sqlite_fdw` accepts the following table-level options via the `CREATE FOREIGN TABLE` command: -- **table** as *string*, **required** +- **table** as *string*, optioanl - SQLite table name. + SQLite table name. Use if not equal to name of foreign table in PostgreSQL. Also see about [identifier case handling](#identifier-case-handling). - **truncatable** as *boolean*, optional @@ -154,7 +154,7 @@ Just ensure `postgres` OS user have permissions for reading or writing on SQLite - **column_name** as *string*, optional - This option gives the column name to use for the column on the remote server. + This option gives the column name to use for the column on the remote server. Also see about [identifier case handling](#identifier-case-handling). - **column_type** as *string*, optional @@ -177,11 +177,11 @@ from PostgreSQL 14. As SQlite does not provide a `TRUNCATE` command, it is simulated with a simple unqualified `DELETE` operation. -Following restrictions apply: +Actually, `TRUNCATE ... CASCADE` can be simulated if we create child table of SQLite with foreign keys and ON DELETE CASCADE, and then executing TRUNCATE (which will be deparsed to DELETE). - - `TRUNCATE ... CASCADE` is not supported +Following restrictions apply: - `TRUNCATE ... RESTART IDENTITY` is not supported - - SQLlite tables with foreign key references can cause errors during truncating + - SQLite tables with foreign key references can cause errors during truncating Functions --------- From bfdb849ec03584768c51077c545c98ac7144b7e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 10:47:46 +0300 Subject: [PATCH 08/22] Fix README.md SQL keywords --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1a9b5a85..f8325f77 100644 --- a/README.md +++ b/README.md @@ -52,8 +52,8 @@ Features ### Notes about features - SQLite evaluates division by zero as `NULL`. It is different from PostgreSQL, which will display `Division by zero` error. -- The data type of column of foreign table should match with data type of column in SQLite to avoid wrong result. For example, if the column of SQLite is `float` (which will be stored as `float8`), the column of foreign table should be `float8`, too. If the column of foreign table is `float4`, it may cause wrong result when select. -- For `key` option, user needs to specify the primary key column of SQLite table corresponding with the `key` option. If not, wrong result may occur when update or delete. +- The data type of column of foreign table should match with data type of column in SQLite to avoid wrong result. For example, if the column of SQLite is `float` (which will be stored as `float8`), the column of foreign table should be `float8`, too. If the column of foreign table is `float4`, it may cause wrong result when `SELECT`. +- For `key` option, user needs to specify the primary key column of SQLite table corresponding with the `key` option. If not, wrong result may occur when `UPDATE` or `DELETE`. - When `Sum` of data in table is out of range, `sqlite_fdw` will display `Infinity` value. It is different from PostgreSQL FDW, which will display `ERROR: value out of range: overflow` error. - For `numeric` data type, `sqlite_fdw` use `sqlite3_column_double` to get value, while SQLite shell uses `sqlite3_column_text` to get value. Those 2 APIs may return different numeric value. Therefore, for `numeric` data type, the value returned from `sqlite_fdw` may different from the value returned from SQLite shell. - `sqlite_fdw` can return implementation-dependent order for column if the column is not specified in `ORDER BY` clause. @@ -177,7 +177,7 @@ from PostgreSQL 14. As SQlite does not provide a `TRUNCATE` command, it is simulated with a simple unqualified `DELETE` operation. -Actually, `TRUNCATE ... CASCADE` can be simulated if we create child table of SQLite with foreign keys and ON DELETE CASCADE, and then executing TRUNCATE (which will be deparsed to DELETE). +Actually, `TRUNCATE ... CASCADE` can be simulated if we create child table of SQLite with foreign keys and `ON DELETE CASCADE`, and then executing `TRUNCATE` (which will be deparsed to `DELETE`). Following restrictions apply: - `TRUNCATE ... RESTART IDENTITY` is not supported From 88a573f98c0f2235542e22199b7b5b2a1084092e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 12:28:47 +0300 Subject: [PATCH 09/22] Fix README.md indents --- README.md | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index f8325f77..e886a34d 100644 --- a/README.md +++ b/README.md @@ -216,12 +216,20 @@ This SQL isn't correct for SQLite: `Error: duplicate column name: a`, but is cor For SQLite there is no difference between ```sql - SELECT * FROM t; - SELECT * FROM T; - SELECT * FROM "t"; - SELECT * FROM "T"; + SELECT * + FROM t; + SELECT * + FROM T; + SELECT * + FROM "t"; + SELECT * + FROM "T"; ``` -For PostgreSQL `SELECT * FROM "T";` is independend query to table `T`, not to table `t` as first queries. +For PostgreSQL +```sql + SELECT * + FROM "T"; +``` is independend query to table `T`, not to table `t` as first queries from previous example. If there is @@ -287,9 +295,9 @@ Please specify SQLite database path using `database` option. ```sql CREATE SERVER sqlite_server FOREIGN DATA WRAPPER sqlite_fdw - OPTIONS ( - database '/path/to/database' - ); + OPTIONS ( + database '/path/to/database' + ); ``` ### No need user mapping From b19712e9d698268634446aa04de92abebadf05ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 12:39:49 +0300 Subject: [PATCH 10/22] README.md, add about USER MAPPING In more universal format --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index e886a34d..58b17736 100644 --- a/README.md +++ b/README.md @@ -127,9 +127,9 @@ Usage ## CREATE USER MAPPING options -`sqlite_fdw` no need any `CREATE USER MAPPING` command. +There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. -Just ensure `postgres` OS user have permissions for reading or writing on SQLite database file. +Just ensure `postgres` OS user have permissions for reading or writing on SQLite database file. There is no other access problem than OS access permissions to SQLite database file. ## CREATE FOREIGN TABLE options @@ -300,7 +300,9 @@ Please specify SQLite database path using `database` option. ); ``` -### No need user mapping +### User mapping + +There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. ### Create foreign table Please specify `table` option if SQLite table name is different from foreign table name. From 72bbf702f0261ec9db01bf41c578c69bef2f68f5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 12:41:59 +0300 Subject: [PATCH 11/22] README.md, fix SQL quoting --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 58b17736..695f25d7 100644 --- a/README.md +++ b/README.md @@ -229,7 +229,8 @@ For PostgreSQL ```sql SELECT * FROM "T"; -``` is independend query to table `T`, not to table `t` as first queries from previous example. +``` +is independend query to table `T`, not to table `t` as first queries from previous example. If there is From 5942c1ca9f21251d60cbb531a3960dc5bb9a643c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 12:45:29 +0300 Subject: [PATCH 12/22] README.md add about SQLite version. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 695f25d7..be03bb37 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ SQLite Foreign Data Wrapper for PostgreSQL ========================================== This is a foreign data wrapper (FDW) to connect [PostgreSQL](https://www.postgresql.org/) -to [SQLite](https://sqlite.org/) database file. This FDW works with PostgreSQL 11, 12, 13, 14 and 15. +to [SQLite](https://sqlite.org/) database file. This FDW works with PostgreSQL 11, 12, 13, 14, 15 and confirmed with SQLite 3.38.5. Contents -------- From 3013f68db9063ddb4137c49b8d36d7fc4e5af851 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 30 Jan 2023 13:57:19 +0300 Subject: [PATCH 13/22] README.md, many fixes --- README.md | 79 +++++++++++++++++++++++++++++++++---------------------- 1 file changed, 47 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 9687d0b8..dbc92f7a 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ SQLite Foreign Data Wrapper for PostgreSQL This is a foreign data wrapper (FDW) to connect [PostgreSQL](https://www.postgresql.org/) to [SQLite](https://sqlite.org/) database file. This FDW works with PostgreSQL 11, 12, 13, 14, 15 and confirmed with SQLite 3.38.5. +PostgreSQL + SQLite + Contents -------- @@ -137,7 +139,7 @@ Just ensure `postgres` OS user have permissions for reading or writing on SQLite `sqlite_fdw` accepts the following table-level options via the `CREATE FOREIGN TABLE` command: -- **table** as *string*, optioanl +- **table** as *string*, optional SQLite table name. Use if not equal to name of foreign table in PostgreSQL. Also see about [identifier case handling](#identifier-case-handling). @@ -216,21 +218,12 @@ This SQL isn't correct for SQLite: `Error: duplicate column name: a`, but is cor For SQLite there is no difference between ```sql - SELECT * - FROM t; - SELECT * - FROM T; - SELECT * - FROM "t"; - SELECT * - FROM "T"; -``` -For PostgreSQL -```sql - SELECT * - FROM "T"; + SELECT * FROM t; -- №1 + SELECT * FROM T; -- №2 + SELECT * FROM "t"; -- №3 + SELECT * FROM "T"; -- №4 ``` -is independend query to table `T`, not to table `t` as first queries from previous example. +For PostgreSQL the query with comment `№4` is independend query to table `T`, not to table `t` as other queries. If there is @@ -240,7 +233,7 @@ If there is b REAL ); ``` -in SQLite, both `a` and `A` , `b` and `B` columns will have the same real datasource in SQlite +in SQLite, both `a` and `A` , `b` and `B` columns will have the same real datasource in SQlite in follow foreign table: ```sql CREATE FOREIGN TABLE "SQLite test" ( @@ -259,7 +252,7 @@ Generated columns SQLite provides support for [generated columns](https://www.sqlite.org/gencol.html). Behaviour of `sqlite_fdw` with this columns _isn't yet described_. -Note that while `sqlite_fdw` will `insert` or `update` the generated column value +Note that while `sqlite_fdw` will `INSERT` or `UPDATE` the generated column value in SQLite, there is nothing to stop the value being modified within SQLite, and hence no guarantee that in subsequent `SELECT` operations the column will still contain the expected generated value. This limitation also applies to @@ -285,33 +278,47 @@ Examples ### Install the extension: +Once for a database you need, as PostgreSQL superuser. + ```sql CREATE EXTENSION sqlite_fdw; ``` ### Create a foreign server with appropriate configuration: -Please specify SQLite database path using `database` option. +Once for a foreign datasource you need, as PostgreSQL superuser. Please specify SQLite database path using `database` option. ```sql CREATE SERVER sqlite_server - FOREIGN DATA WRAPPER sqlite_fdw - OPTIONS ( - database '/path/to/database' - ); + FOREIGN DATA WRAPPER sqlite_fdw + OPTIONS ( + database '/path/to/database' + ); +``` + +### Grant usage on foreign server to normal user in PostgreSQL: + +Once for a normal user (non-superuser) in PostgreSQL, as PostgreSQL superuser. It is a good idea to use a superuser only where really necessary, so let's allow a normal user to use the foreign server (this is not required for the example to work, but it's secirity recomedation). + +```sql + GRANT USAGE ON FOREIGN SERVER sqlite_server TO pguser; ``` +Where `pguser` is a sample user for works with foreign server (and foreign tables). ### User mapping There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. ### Create foreign table +All `CREATE FOREIGN TABLE` SQL commands can be executed as a normal PostgreSQL user if there were correct `GRANT USAGE ON FOREIGN SERVER`. No need PostgreSQL supersuer for secirity reasons but also works with PostgreSQL supersuer. + Please specify `table` option if SQLite table name is different from foreign table name. ```sql CREATE FOREIGN TABLE t1 ( a integer, - b text) + b text + ) SERVER sqlite_server OPTIONS ( table 't1_sqlite' @@ -323,7 +330,8 @@ If you want to update tables, please add `OPTIONS (key 'true')` to a primary key ```sql CREATE FOREIGN TABLE t1( a integer OPTIONS (key 'true'), - b text) + b text + ) SERVER sqlite_server OPTIONS ( table 't1_sqlite' @@ -336,7 +344,8 @@ If you need to convert INT SQLite column (epoch Unix Time) to be treated/visuali CREATE FOREIGN TABLE t1( a integer, b text, - c timestamp without time zone OPTIONS (column_type 'INT')) + c timestamp without time zone OPTIONS (column_type 'INT') + ) SERVER sqlite_server OPTIONS ( table 't1_sqlite' @@ -350,6 +359,7 @@ As above, but with aliased column names: a integer, b text OPTIONS (column_name 'test_id'), c timestamp without time zone OPTIONS (column_type 'INT', column_name 'unixtime') + ) SERVER sqlite_server OPTIONS ( table 't1_sqlite' @@ -360,8 +370,8 @@ As above, but with aliased column names: ```sql IMPORT FOREIGN SCHEMA someschema - FROM SERVER sqlite_server; - INTO public; + FROM SERVER sqlite_server + INTO public; ``` Note: `someschema` has no particular meaning and can be set to an arbitrary value. @@ -370,19 +380,25 @@ Note: `someschema` has no particular meaning and can be set to an arbitrary valu For the table from previous examples ```sql - SELECT * - FROM t1; + SELECT * FROM t1; ``` Limitations ----------- +### SQL commands - `COPY` command for foreign tables is not supported - `IMPORT` of generated column is not supported -- Insert into a partitioned table which has foreign partitions is not supported. Error "Not support partition insert" will display. +- `INSERT` into a partitioned table which has foreign partitions is not supported. Error `Not support partition insert` will display. - `TRUNCATE` in `sqlite_fdw` always delete data of both parent and child tables (no matter user inputs `TRUNCATE table CASCADE` or `TRUNCATE table RESTRICT`) if there are foreign-keys references with `ON DELETE CASCADE` clause. - `RETURNING` is not supported. -- `sqlite_fdw` only supports `ARRAY` const, for example, `ANY (ARRAY[1, 2, 3])` or `ANY ('{1, 2 ,3}')`. `Sqlite_fdw` does not support `ARRAY` expression, for example, `ANY (ARRAY[c1, 1, c1+0])`. For `ANY(ARRAY)` clause, `sqlite_fdw` deparses it using `IN` operator. + +### Arrays +- `sqlite_fdw` only supports `ARRAY` const, for example, `ANY (ARRAY[1, 2, 3])` or `ANY ('{1, 2 ,3}')`. +- `sqlite_fdw` does not support `ARRAY` expression, for example, `ANY (ARRAY[c1, 1, c1+0])`. +- For `ANY(ARRAY)` clause, `sqlite_fdw` deparses it using `IN` operator. + +### Numbers (range and precision) - For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. - SQLite does not support `numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. @@ -394,7 +410,6 @@ Tests Contributing ------------ -## Contributing Opening issues and pull requests on GitHub are welcome. Useful links From c56c816ef61f4eadb3949d1b7a0f79cff850aa6a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Fri, 3 Feb 2023 17:57:52 +0300 Subject: [PATCH 14/22] README.md, add about special values IEEE 754-2008 and booleans --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index dbc92f7a..3524948a 100644 --- a/README.md +++ b/README.md @@ -402,6 +402,11 @@ Limitations - For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. - SQLite does not support `numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. +- SQLite does not support special values for IEEE 754-2008 numbers stored in SQLite `real` [affinity](https://www.sqlite.org/datatype3.html) in opposite PostgreSQL with full support of special values. This values are `NaN` ≡ ` 0.0 / 0.0 `, `+Infinity` ≡ +∞ ≡ ` 1.0 / 0.0 `,`-Infinity` ≡ -∞ ≡ ` -1.0 / 0.0 `. All this special calculating expressions gives `NULL` in SQLite in opposite to full-supported special values in PostgreSQL. `sqlite_fdw` can case insensetive read special values for this pseudonumbers from data with `text` affinity in SQLite such as `NaN`, `nan`, `Nan`, `+INF`, `Infinity`, `-Inf` etc and convert it to special values in PostgreSQL. But during `SELECT ... WHERE ... = double precision 'naN'` or other query reverse conversion `sqlite_fdw` uses **only** standard PostgreSQL literals: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE`. *This can caused selecting issues*. + +### Boolean values +- `sqlite_fdw` boolean values support exists only for `bool` columns in foreign table. SQLite documentation recommends to store boolean as value with `integer` [affinity](https://www.sqlite.org/datatype3.html). `NULL` isn't converted, 1 converted to `true`, all other `NON NULL` values converted to `false`. During `SELECT ... WHERE condition_column` condition converted only to `condition_column = 1` and `SELECT ... WHERE NOT condition_column` converted only to `condition_column != 1`. +- `sqlite_fdw` provides limited support of boolean values if `bool` column in foreign table mapped to SQLite `text` [affinity](https://www.sqlite.org/datatype3.html). Case insensetive values `t`, `true`, `y`, `yes` converted to `true` value, all other `NOT NULL` values converted to `false`. De facto `SELECT ... WHERE condition_column` queries isn't applicable to this case, because search condition for `bool` column coverted by `sqlite_fdw` **only** to `condition_column = 1`. *This can caused selecting issues*. Tests ----- From d0d04713a23199c8bfec7aa358e91df915a65c58 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Sat, 4 Feb 2023 23:31:58 +0300 Subject: [PATCH 15/22] README.md Add about OS permissions --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3524948a..3f32d835 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,7 @@ Usage There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. -Just ensure `postgres` OS user have permissions for reading or writing on SQLite database file. There is no other access problem than OS access permissions to SQLite database file. +Just ensure `postgres` OS user have permissions for reading on SQLite database file or also for writing on **both** database file and directory near this file. There is no other access problem than OS access permissions to SQLite database file and write permissoins for directory of this file. For `INSERT`, `UPDATE`, `DELETE` and other changes in SQLite database from PostgreSQL ensure not only SQLite databse file permissions, but **also writing permissons of directory of SQLite database** file for `postgres` OS user. Otherwise error `failed to execute remote SQL: rc=8 attempt to write a readonly database` occurs. During data change transactions SQLite create some temporary files in direcory near SQLite database file. ## CREATE FOREIGN TABLE options From dbaacc517bab6ced9f87666c454ebbc25c4cfc60 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Sun, 5 Feb 2023 18:58:54 +0300 Subject: [PATCH 16/22] Fix README.md --- CHANGELOG | 25 +++++++++++++++++++++++++ README.md | 19 +++++++++++++------ 2 files changed, 38 insertions(+), 6 deletions(-) create mode 100644 CHANGELOG diff --git a/CHANGELOG b/CHANGELOG new file mode 100644 index 00000000..24e65c99 --- /dev/null +++ b/CHANGELOG @@ -0,0 +1,25 @@ +Revision history for sqlite_fdw + +2.2.0 2022-09-22 + - Changes yet not described + +2.1.1 2021-12-22 + +2.1.0 2021-09-24 + +2.0.0 2021-05-26 + +1.3.1 2021-01-14 + +1.3.0 2020-09-24 + +1.2.1 2020-08-26 + +1.2.0 2019-10-01 + +1.1.1 2019-08-26 + +1.1.0 2018-10-31 + +0.0.0 2018-04-09 + - Initial commit diff --git a/README.md b/README.md index 3f32d835..a3ffa76b 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,8 @@ to [SQLite](https://sqlite.org/) database file. This FDW works with PostgreSQL 1 PostgreSQL + SQLite +History of releases and main features see in [CHANGELOG](CHANGELOG). + Contents -------- @@ -125,14 +127,19 @@ Usage - **batch_size** as *integer*, optional Specifies the number of rows which should be inserted in a single `INSERT` operation. This setting can be overridden for individual tables. + +- *planned feature* **readonly** as *boolean*, optional + This option disallow write operations on SQLite database file. Uses `SQLITE_OPEN_READONLY` mode. ## CREATE USER MAPPING options There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. -Just ensure `postgres` OS user have permissions for reading on SQLite database file or also for writing on **both** database file and directory near this file. There is no other access problem than OS access permissions to SQLite database file and write permissoins for directory of this file. For `INSERT`, `UPDATE`, `DELETE` and other changes in SQLite database from PostgreSQL ensure not only SQLite databse file permissions, but **also writing permissons of directory of SQLite database** file for `postgres` OS user. Otherwise error `failed to execute remote SQL: rc=8 attempt to write a readonly database` occurs. During data change transactions SQLite create some temporary files in direcory near SQLite database file. - +In OS `sqlite_fdw` works as executed code with permissions of user of PostgreSQL server. Usually it is `postgres` OS user. For interacting with SQLite database without access errors ensure this user have follow permissions: +- read permission on all directories by path to the SQLite database file; +- read permission on SQLite database file; +- write permissions both on SQLite database file and *directory it contains* if you don't need *readonly* mode. During `INSERT`, `UPDATE` or `DELETE` in SQLite database, SQLite engine functions makes temporary files with transaction data in the directory near SQLite database file. Hence without write permissions you'll have a message `failed to execute remote SQL: rc=8 attempt to write a readonly database`. ## CREATE FOREIGN TABLE options @@ -307,7 +314,7 @@ Where `pguser` is a sample user for works with foreign server (and foreign table ### User mapping -There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. +There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. About access problems see in [CREATE USER MAPPING options](#create-user-mapping-options). ### Create foreign table All `CREATE FOREIGN TABLE` SQL commands can be executed as a normal PostgreSQL user if there were correct `GRANT USAGE ON FOREIGN SERVER`. No need PostgreSQL supersuer for secirity reasons but also works with PostgreSQL supersuer. @@ -402,11 +409,11 @@ Limitations - For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. - SQLite does not support `numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. -- SQLite does not support special values for IEEE 754-2008 numbers stored in SQLite `real` [affinity](https://www.sqlite.org/datatype3.html) in opposite PostgreSQL with full support of special values. This values are `NaN` ≡ ` 0.0 / 0.0 `, `+Infinity` ≡ +∞ ≡ ` 1.0 / 0.0 `,`-Infinity` ≡ -∞ ≡ ` -1.0 / 0.0 `. All this special calculating expressions gives `NULL` in SQLite in opposite to full-supported special values in PostgreSQL. `sqlite_fdw` can case insensetive read special values for this pseudonumbers from data with `text` affinity in SQLite such as `NaN`, `nan`, `Nan`, `+INF`, `Infinity`, `-Inf` etc and convert it to special values in PostgreSQL. But during `SELECT ... WHERE ... = double precision 'naN'` or other query reverse conversion `sqlite_fdw` uses **only** standard PostgreSQL literals: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE`. *This can caused selecting issues*. +- SQLite does not support special values for IEEE 754-2008 numbers stored in SQLite `real` [affinity](https://www.sqlite.org/datatype3.html) in opposite PostgreSQL with full support of special values. In PostgreSQL there is full support of such special values as `NaN` ≡ ` 0.0 / 0.0 `, `+Infinity` ≡ +∞ ≡ ` 1.0 / 0.0 `,`-Infinity` ≡ -∞ ≡ ` -1.0 / 0.0 `. In oppose to PostgreSQL all this special calculating expressions gives `NULL` in SQLite. Also conditions with special literals (such as ` n < '+Infinity'` or ` m > '-Infinity'` ) isn't numeric conditions in SQLite and gives unexpected result after pushdowning in oppose to internal PostgreSQL calculations. During `INSERT` or in `WHERE` conditions `sqlite_fdw` uses given by PostgreSQL standard case sensetive literals **only** in follow forms: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE` condition. *This can caused selecting issues*. ### Boolean values -- `sqlite_fdw` boolean values support exists only for `bool` columns in foreign table. SQLite documentation recommends to store boolean as value with `integer` [affinity](https://www.sqlite.org/datatype3.html). `NULL` isn't converted, 1 converted to `true`, all other `NON NULL` values converted to `false`. During `SELECT ... WHERE condition_column` condition converted only to `condition_column = 1` and `SELECT ... WHERE NOT condition_column` converted only to `condition_column != 1`. -- `sqlite_fdw` provides limited support of boolean values if `bool` column in foreign table mapped to SQLite `text` [affinity](https://www.sqlite.org/datatype3.html). Case insensetive values `t`, `true`, `y`, `yes` converted to `true` value, all other `NOT NULL` values converted to `false`. De facto `SELECT ... WHERE condition_column` queries isn't applicable to this case, because search condition for `bool` column coverted by `sqlite_fdw` **only** to `condition_column = 1`. *This can caused selecting issues*. +- `sqlite_fdw` boolean values support exists only for `bool` columns in foreign table. SQLite documentation recommends to store boolean as value with `integer` [affinity](https://www.sqlite.org/datatype3.html). `NULL` isn't converted, 1 converted to `true`, all other `NON NULL` values converted to `false`. During `SELECT ... WHERE condition_column` condition converted only to `condition_column`. +- `sqlite_fdw` don't provides limited support of boolean values if `bool` column in foreign table mapped to SQLite `text` [affinity](https://www.sqlite.org/datatype3.html). Tests ----- From 6112b4e47077ab0706ef39e60c62c36ac4036a28 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 6 Feb 2023 08:31:39 +0300 Subject: [PATCH 17/22] CHANGELOG Add some history points --- CHANGELOG | 55 ++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 17 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 24e65c99..1a50a0f6 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,25 +1,46 @@ Revision history for sqlite_fdw 2.2.0 2022-09-22 - - Changes yet not described + - Working with busy connections 2.1.1 2021-12-22 2.1.0 2021-09-24 - -2.0.0 2021-05-26 - -1.3.1 2021-01-14 - -1.3.0 2020-09-24 - -1.2.1 2020-08-26 - -1.2.0 2019-10-01 - -1.1.1 2019-08-26 - -1.1.0 2018-10-31 - -0.0.0 2018-04-09 + - support for PostgreSQL 14 + +2.0.0 2021-05-26 + - Support JOIN pushdown (LEFT,RIGHT,INNER) + - Support direct modification (UPDATE/DELETE) + - Support pushdown nest functions + - Support pushdown scalar operator ANY/ALL (ARRAY) + - Support pushdown ON CONFLICT DO NOTHING + - Refactored tests + - Don't push down lower/upper function + - Fix processing for DATE data type + - Do not prepare SQL statement during EXPLAIN + - add sqlite_fdw_version() + +1.3.1 2021-01-14 + +1.3.0 2020-09-24 + - column_type option + +1.2.1 2020-08-26 + - pushing down limit/order clauses + - support NULLS FIRST/LAST + +1.2.0 2019-10-01 + - many fixes of different errors + +1.1.1 2019-08-26 + - column_name option + - key option + +1.1.0 2018-10-31 + - Transactions + - Support update to foreign table + - WHERE clauses are pushdowned + - Aggregate function are pushdowned + +0.0.0 2018-04-09 - Initial commit From 3cafb2f581371914fc75a02bab7fb0728ed8ff0e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 6 Feb 2023 10:45:25 +0300 Subject: [PATCH 18/22] README.md, by review --- README.md | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index a3ffa76b..d4f725c0 100644 --- a/README.md +++ b/README.md @@ -128,10 +128,6 @@ Usage Specifies the number of rows which should be inserted in a single `INSERT` operation. This setting can be overridden for individual tables. -- *planned feature* **readonly** as *boolean*, optional - - This option disallow write operations on SQLite database file. Uses `SQLITE_OPEN_READONLY` mode. - ## CREATE USER MAPPING options There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. @@ -139,7 +135,7 @@ There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need a In OS `sqlite_fdw` works as executed code with permissions of user of PostgreSQL server. Usually it is `postgres` OS user. For interacting with SQLite database without access errors ensure this user have follow permissions: - read permission on all directories by path to the SQLite database file; - read permission on SQLite database file; -- write permissions both on SQLite database file and *directory it contains* if you don't need *readonly* mode. During `INSERT`, `UPDATE` or `DELETE` in SQLite database, SQLite engine functions makes temporary files with transaction data in the directory near SQLite database file. Hence without write permissions you'll have a message `failed to execute remote SQL: rc=8 attempt to write a readonly database`. +- write permissions both on SQLite database file and *directory it contains* if you need a modification. During `INSERT`, `UPDATE` or `DELETE` in SQLite database, SQLite engine functions makes temporary files with transaction data in the directory near SQLite database file. Hence without write permissions you'll have a message `failed to execute remote SQL: rc=8 attempt to write a readonly database`. ## CREATE FOREIGN TABLE options @@ -314,7 +310,7 @@ Where `pguser` is a sample user for works with foreign server (and foreign table ### User mapping -There is no user or password conceptions in SQlite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. About access problems see in [CREATE USER MAPPING options](#create-user-mapping-options). +There is no user or password conceptions in SQLite, hence `sqlite_fdw` no need any `CREATE USER MAPPING` command. About access problems see in [CREATE USER MAPPING options](#create-user-mapping-options). ### Create foreign table All `CREATE FOREIGN TABLE` SQL commands can be executed as a normal PostgreSQL user if there were correct `GRANT USAGE ON FOREIGN SERVER`. No need PostgreSQL supersuer for secirity reasons but also works with PostgreSQL supersuer. @@ -409,7 +405,7 @@ Limitations - For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. - SQLite does not support `numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. -- SQLite does not support special values for IEEE 754-2008 numbers stored in SQLite `real` [affinity](https://www.sqlite.org/datatype3.html) in opposite PostgreSQL with full support of special values. In PostgreSQL there is full support of such special values as `NaN` ≡ ` 0.0 / 0.0 `, `+Infinity` ≡ +∞ ≡ ` 1.0 / 0.0 `,`-Infinity` ≡ -∞ ≡ ` -1.0 / 0.0 `. In oppose to PostgreSQL all this special calculating expressions gives `NULL` in SQLite. Also conditions with special literals (such as ` n < '+Infinity'` or ` m > '-Infinity'` ) isn't numeric conditions in SQLite and gives unexpected result after pushdowning in oppose to internal PostgreSQL calculations. During `INSERT` or in `WHERE` conditions `sqlite_fdw` uses given by PostgreSQL standard case sensetive literals **only** in follow forms: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE` condition. *This can caused selecting issues*. +- SQLite does not support special values for IEEE 754-2008 numbers stored in SQLite `real` [affinity](https://www.sqlite.org/datatype3.html) in opposite PostgreSQL with full support of special values. In PostgreSQL there is full support of such special values as `NaN` ≡ C language ` 0.0 / 0.0 `, `+Infinity` ≡ +∞ ≡ C language ` 1.0 / 0.0 `,`-Infinity` ≡ -∞ ≡ C language ` -1.0 / 0.0 `. In oppose to PostgreSQL, SQLite stores them as a text value. Also conditions with special literals (such as ` n < '+Infinity'` or ` m > '-Infinity'` ) isn't numeric conditions in SQLite and gives unexpected result after pushdowning in oppose to internal PostgreSQL calculations. During `INSERT INTO ... SELECT` or in `WHERE` conditions `sqlite_fdw` uses given by PostgreSQL standard case sensetive literals **only** in follow forms: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE` condition. *This can caused selecting issues*. ### Boolean values - `sqlite_fdw` boolean values support exists only for `bool` columns in foreign table. SQLite documentation recommends to store boolean as value with `integer` [affinity](https://www.sqlite.org/datatype3.html). `NULL` isn't converted, 1 converted to `true`, all other `NON NULL` values converted to `false`. During `SELECT ... WHERE condition_column` condition converted only to `condition_column`. From 4fe3a025216675a5ddd52c6010bd014b38312973 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 6 Feb 2023 10:46:35 +0300 Subject: [PATCH 19/22] Delete CHANGELOG --- CHANGELOG | 46 ---------------------------------------------- 1 file changed, 46 deletions(-) delete mode 100644 CHANGELOG diff --git a/CHANGELOG b/CHANGELOG deleted file mode 100644 index 1a50a0f6..00000000 --- a/CHANGELOG +++ /dev/null @@ -1,46 +0,0 @@ -Revision history for sqlite_fdw - -2.2.0 2022-09-22 - - Working with busy connections - -2.1.1 2021-12-22 - -2.1.0 2021-09-24 - - support for PostgreSQL 14 - -2.0.0 2021-05-26 - - Support JOIN pushdown (LEFT,RIGHT,INNER) - - Support direct modification (UPDATE/DELETE) - - Support pushdown nest functions - - Support pushdown scalar operator ANY/ALL (ARRAY) - - Support pushdown ON CONFLICT DO NOTHING - - Refactored tests - - Don't push down lower/upper function - - Fix processing for DATE data type - - Do not prepare SQL statement during EXPLAIN - - add sqlite_fdw_version() - -1.3.1 2021-01-14 - -1.3.0 2020-09-24 - - column_type option - -1.2.1 2020-08-26 - - pushing down limit/order clauses - - support NULLS FIRST/LAST - -1.2.0 2019-10-01 - - many fixes of different errors - -1.1.1 2019-08-26 - - column_name option - - key option - -1.1.0 2018-10-31 - - Transactions - - Support update to foreign table - - WHERE clauses are pushdowned - - Aggregate function are pushdowned - -0.0.0 2018-04-09 - - Initial commit From 9cd90f834153bd8e08a4793e13ff104edbc49483 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Mon, 6 Feb 2023 10:47:00 +0300 Subject: [PATCH 20/22] Remove CHANGELOG --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index d4f725c0..d21886b5 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,6 @@ to [SQLite](https://sqlite.org/) database file. This FDW works with PostgreSQL 1 PostgreSQL + SQLite -History of releases and main features see in [CHANGELOG](CHANGELOG). - Contents -------- From 4154f2dc52479d7cc3dfb4e4835c3d4985c1bf24 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Tue, 7 Feb 2023 06:33:35 +0300 Subject: [PATCH 21/22] Fix about `NaN`, `+Infinity` and `-Infinity` --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d21886b5..32ce724c 100644 --- a/README.md +++ b/README.md @@ -403,7 +403,7 @@ Limitations - For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. - SQLite does not support `numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. -- SQLite does not support special values for IEEE 754-2008 numbers stored in SQLite `real` [affinity](https://www.sqlite.org/datatype3.html) in opposite PostgreSQL with full support of special values. In PostgreSQL there is full support of such special values as `NaN` ≡ C language ` 0.0 / 0.0 `, `+Infinity` ≡ +∞ ≡ C language ` 1.0 / 0.0 `,`-Infinity` ≡ -∞ ≡ C language ` -1.0 / 0.0 `. In oppose to PostgreSQL, SQLite stores them as a text value. Also conditions with special literals (such as ` n < '+Infinity'` or ` m > '-Infinity'` ) isn't numeric conditions in SQLite and gives unexpected result after pushdowning in oppose to internal PostgreSQL calculations. During `INSERT INTO ... SELECT` or in `WHERE` conditions `sqlite_fdw` uses given by PostgreSQL standard case sensetive literals **only** in follow forms: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE` condition. *This can caused selecting issues*. +- SQLite does not support special values for IEEE 754-2008 numbers such as `NaN`, `+Infinity` and `-Infinity` in SQL expressions with numeric context. Also SQLite can not store this values in SQLite with `real` [affinity](https://www.sqlite.org/datatype3.html). In opposite to SQLite, PostgreSQL can store this values in columns belongs to `real` datatype family such as `float` or `double precision` and use arithmetic comparation for this values. In oppose to PostgreSQL, SQLite stores `NaN`, `+Infinity` and `-Infinity` as a text values. Also conditions with special literals (such as ` n < '+Infinity'` or ` m > '-Infinity'` ) isn't numeric conditions in SQLite and gives unexpected result after pushdowning in oppose to internal PostgreSQL calculations. During `INSERT INTO ... SELECT` or in `WHERE` conditions `sqlite_fdw` uses given by PostgreSQL standard case sensetive literals **only** in follow forms: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE` condition. *This can caused selecting issues*. ### Boolean values - `sqlite_fdw` boolean values support exists only for `bool` columns in foreign table. SQLite documentation recommends to store boolean as value with `integer` [affinity](https://www.sqlite.org/datatype3.html). `NULL` isn't converted, 1 converted to `true`, all other `NON NULL` values converted to `false`. During `SELECT ... WHERE condition_column` condition converted only to `condition_column`. From b6e8660666faf611f72741e1c7b52f3f137bf799 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB?= <41448637+mkgrgis@users.noreply.github.com> Date: Tue, 7 Feb 2023 06:40:19 +0300 Subject: [PATCH 22/22] Fix about special values --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 32ce724c..6f6ff07c 100644 --- a/README.md +++ b/README.md @@ -403,7 +403,7 @@ Limitations - For `sum` function of SQLite, output of `sum(bigint)` is `integer` value. If input values are big, the overflow error may occurs on SQLite because it overflow within the range of signed 64bit. For PostgreSQL, it can calculate as over the precision of `bigint`, so overflow does not occur. - SQLite promises to preserve the 15 most significant digits of a floating point value. The big value which exceed 15 most significant digits may become different value after inserted. - SQLite does not support `numeric` type as PostgreSQL. Therefore, it does not allow to store numbers with too high precision and scale. Error out of range occurs. -- SQLite does not support special values for IEEE 754-2008 numbers such as `NaN`, `+Infinity` and `-Infinity` in SQL expressions with numeric context. Also SQLite can not store this values in SQLite with `real` [affinity](https://www.sqlite.org/datatype3.html). In opposite to SQLite, PostgreSQL can store this values in columns belongs to `real` datatype family such as `float` or `double precision` and use arithmetic comparation for this values. In oppose to PostgreSQL, SQLite stores `NaN`, `+Infinity` and `-Infinity` as a text values. Also conditions with special literals (such as ` n < '+Infinity'` or ` m > '-Infinity'` ) isn't numeric conditions in SQLite and gives unexpected result after pushdowning in oppose to internal PostgreSQL calculations. During `INSERT INTO ... SELECT` or in `WHERE` conditions `sqlite_fdw` uses given by PostgreSQL standard case sensetive literals **only** in follow forms: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE` condition. *This can caused selecting issues*. +- SQLite does not support special values for IEEE 754-2008 numbers such as `NaN`, `+Infinity` and `-Infinity` in SQL expressions with numeric context. Also SQLite can not store this values with `real` [affinity](https://www.sqlite.org/datatype3.html). In opposite to SQLite, PostgreSQL can store special values in columns belongs to `real` datatype family such as `float` or `double precision` and use arithmetic comparation for this values. In oppose to PostgreSQL, SQLite stores `NaN`, `+Infinity` and `-Infinity` as a text values. Also conditions with special literals (such as ` n < '+Infinity'` or ` m > '-Infinity'` ) isn't numeric conditions in SQLite and gives unexpected result after pushdowning in oppose to internal PostgreSQL calculations. During `INSERT INTO ... SELECT` or in `WHERE` conditions `sqlite_fdw` uses given by PostgreSQL standard case sensetive literals **only** in follow forms: `NaN`, `-Infinity`, `Infinity`, not original strings from `WHERE` condition. *This can caused selecting issues*. ### Boolean values - `sqlite_fdw` boolean values support exists only for `bool` columns in foreign table. SQLite documentation recommends to store boolean as value with `integer` [affinity](https://www.sqlite.org/datatype3.html). `NULL` isn't converted, 1 converted to `true`, all other `NON NULL` values converted to `false`. During `SELECT ... WHERE condition_column` condition converted only to `condition_column`.