Stack overflow when transacting :db.type/tupleAttrs with a :db.type/ref attr through :db.fn/call

[seepel]

It seems I've run into a stack overflow somewhere deep inside some internals. I'm hoping this might be obvious to someone familiar with the code.

This is the smallest scenariot I've been able to isolate this bug to.

I am modeling an api repository. Providers have a unique name, each provider has a set of services each with a unique name to the provider.

(def conn (d/create-conn {:provider/name {:db/unique :db.unique/identity}
                           :service/provider {:db/cardinality :db.cardinality/one
                                              :db/valueType :db.type/ref}
                           :service/name {:db/cardinality :db.cardinality/one}
                           :service/pk {:db/valueType :db.type/tuple
                                        :db/tupleAttrs [:service/provider :service/name]
                                        :db/unique :db.unique/identity}}))

We'll choose gmail for our test data

(def tx [{:provider/name "googleapis.com"}
          {:service/provider [:provider/name "googleapis.com"]
           :service/name "gmail/v1"}])

I have another transaction that abstracts this through a :db.fn/call

(def tx-call [[:db.fn/call (fn [_db tx] tx) tx]])

Transacting with the call abstraction works against an empty db

(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (out) [#datascript/Datom [1 :provider/name "googleapis.com" 536870913 true]
; (out)  #datascript/Datom [2 :service/provider 1 536870913 true]
; (out)  #datascript/Datom [2 :service/name "gmail/v1" 536870913 true]
; (out)  #datascript/Datom [2 :service/pk [1 "gmail/v1"] 536870913 true]]

Transacting against a non-empty db with duplicate data seems to go fine

  (clojure.pprint/pprint (:tx-data (d/transact! conn tx)))
  ; (out) []

Transacting against a non-empty db with duplicate data overflows

  (clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
  ; (err) Execution error (StackOverflowError) at datascript.db/is-attr? (db.cljc:1161).
  ; (err) null

Transacting against a non-empty db with duplicate data overflows in random spots

  (clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
  ; (err) Execution error (StackOverflowError) at datascript.db$reify__3146/compare (db.cljc:543).
  ; (err) null
  (clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
  ; (err) Execution error (StackOverflowError) at datascript.db$reify__3146/compare (db.cljc:543).
  ; (err) null
  (clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
  ; (err) Execution error (StackOverflowError) at datascript.db$reify__3158/compare (db.cljc:558).
  ; (err) null
  (clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
  ; (err) Execution error (StackOverflowError) at datascript.db/is-attr? (db.cljc:1161).
  ; (err) null
  (clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
  ; (err) Execution error (StackOverflowError) at datascript.db.DB/_search (db.cljc:671).
  ; (err) null

[seepel]

Here is all of that as one code block

(def conn (d/create-conn {:provider/name {:db/unique :db.unique/identity}
                          :service/provider {:db/cardinality :db.cardinality/one
                                             :db/valueType :db.type/ref}
                          :service/name {:db/cardinality :db.cardinality/one}
                          :service/pk {:db/valueType :db.type/tuple
                                       :db/tupleAttrs [:service/provider :service/name]
                                       :db/unique :db.unique/identity}}))
(def tx [{:provider/name "googleapis.com"}
         {:service/provider [:provider/name "googleapis.com"]
          :service/name "gmail/v1"}])
(def tx-call [[:db.fn/call (fn [_db tx] tx) tx]])
;; Transacting the tx with the call with an empty db seems to go fine
(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (out) [#datascript/Datom [1 :provider/name "googleapis.com" 536870913 true]
; (out)  #datascript/Datom [2 :service/provider 1 536870913 true]
; (out)  #datascript/Datom [2 :service/name "gmail/v1" 536870913 true]
; (out)  #datascript/Datom [2 :service/pk [1 "gmail/v1"] 536870913 true]]
                                                                                                 
;; Transacting against a non-empty db with duplicate data seems to go fine
(clojure.pprint/pprint (:tx-data (d/transact! conn tx))) ;; tx-report
; (out) []
                                                                                                 
;; Transacting against a non-empty db with duplicate data overflows
(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (err) Execution error (StackOverflowError) at datascript.db/is-attr? (db.cljc:1161).
; (err) null    
                                                                                                 
;; Transacting against a non-empty db with duplicate data overflows in random spots
(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (err) Execution error (StackOverflowError) at datascript.db$reify__3146/compare (db.cljc:543).
; (err) null
(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (err) Execution error (StackOverflowError) at datascript.db$reify__3146/compare (db.cljc:543).
; (err) null
(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (err) Execution error (StackOverflowError) at datascript.db$reify__3158/compare (db.cljc:558).
; (err) null
(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (err) Execution error (StackOverflowError) at datascript.db/is-attr? (db.cljc:1161).
; (err) null
(clojure.pprint/pprint (:tx-data (d/transact! conn tx-call)))
; (err) Execution error (StackOverflowError) at datascript.db.DB/_search (db.cljc:671).
; (err) null

[tonsky]

Oh wow. A lot of stuff come together for this to fail. There’s a loop in retry-with-tempids that happen when you do upserts. It normally works but here it fails because you call a function that generates new tempids and it can’t detect cycle because of that.