Skip to content

feat: Created Filter feature and redesign Database UI #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yash-pouranik merged 1 commit into from
Feb 25, 2026
Merged

feat: Created Filter feature and redesign Database UI #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion backend/controllers/data.controller.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ const mongoose = require('mongoose');
const Project = require("../models/Project");
const { getConnection } = require("../utils/connection.manager");
const { getCompiledModel } = require("../utils/injectModel");
const QueryEngine = require("../utils/queryEngine");

// Validate MongoDB ObjectId
const isValidId = (id) => mongoose.Types.ObjectId.isValid(id);
Expand Down Expand Up @@ -82,7 +83,12 @@ module.exports.getAllData = async (req, res) => {
const connection = await getConnection(project._id);
const Model = getCompiledModel(connection, collectionConfig, project._id, project.resources.db.isExternal);

const data = await Model.find({}).limit(100).lean();
const features = new QueryEngine(Model.find(), req.query)
.filter()
.sort()
.paginate();

const data = await features.query.lean();
console.timeEnd("getall")
res.json(data);
} catch (err) {
Expand Down
8 changes: 7 additions & 1 deletion backend/controllers/project.controller.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ const { encrypt } = require('../utils/encryption');
const { URL } = require('url');
const { getConnection } = require("../utils/connection.manager");
const { getCompiledModel } = require("../utils/injectModel")
const QueryEngine = require("../utils/queryEngine");
const { storageRegistry } = require("../utils/registry");
const { deleteProjectByApiKeyCache, setProjectById, getProjectById, deleteProjectById } = require("../services/redisCaching");
const { getPublicIp } = require("../utils/network");
Expand Down Expand Up @@ -332,7 +333,12 @@ module.exports.getData = async (req, res) => {

// const collectionsList = await mongoose.connection.db.listCollections({ name: finalCollectionName }).toArray();

const data = await model.find({}).limit(50);
const features = new QueryEngine(model.find(), req.query)
.filter()
.sort()
.paginate();

const data = await features.query.lean();

// let data = [];
// if (collectionsList.length > 0) {
Expand Down
65 changes: 65 additions & 0 deletions backend/utils/queryEngine.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
class QueryEngine {
constructor(query, queryString) {
this.query = query;
this.queryString = queryString;
}

filter() {
const queryObj = { ...this.queryString };
const excludedFields = ['page', 'sort', 'limit', 'fields'];
excludedFields.forEach(el => delete queryObj[el]);

const mongoQuery = {};
for (const key in queryObj) {
if (key.endsWith('_gt')) {
const field = key.replace(/_gt$/, '');
mongoQuery[field] = { ...mongoQuery[field], $gt: queryObj[key] };
} else if (key.endsWith('_gte')) {
const field = key.replace(/_gte$/, '');
mongoQuery[field] = { ...mongoQuery[field], $gte: queryObj[key] };
} else if (key.endsWith('_lt')) {
const field = key.replace(/_lt$/, '');
mongoQuery[field] = { ...mongoQuery[field], $lt: queryObj[key] };
} else if (key.endsWith('_lte')) {
const field = key.replace(/_lte$/, '');
mongoQuery[field] = { ...mongoQuery[field], $lte: queryObj[key] };
} else {
mongoQuery[key] = queryObj[key];
}
}
Comment on lines +12 to +29

@coderabbitai coderabbitai Bot Feb 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Query values are always strings — numeric comparisons will silently misbehave.

req.query values are always strings. When you build { price: { $gt: "5" } }, MongoDB performs a lexicographic comparison, not a numeric one. This means price_gt=5 won't correctly filter numeric fields (e.g., "50" is less than "5" lexicographically).

You need to coerce values to their appropriate types. At minimum, attempt numeric parsing for comparison operators:

Proposed fix 
+        const coerce = (val) => {
+            if (!isNaN(val) && val.trim() !== '') return Number(val);
+            // Could also handle date parsing here
+            return val;
+        };
+
         const mongoQuery = {};
         for (const key in queryObj) {
             if (key.endsWith('_gt')) {
                 const field = key.replace(/_gt$/, '');
-                mongoQuery[field] = { ...mongoQuery[field], $gt: queryObj[key] };
+                mongoQuery[field] = { ...mongoQuery[field], $gt: coerce(queryObj[key]) };
             } else if (key.endsWith('_gte')) {
                 const field = key.replace(/_gte$/, '');
-                mongoQuery[field] = { ...mongoQuery[field], $gte: queryObj[key] };
+                mongoQuery[field] = { ...mongoQuery[field], $gte: coerce(queryObj[key]) };
             } else if (key.endsWith('_lt')) {
                 const field = key.replace(/_lt$/, '');
-                mongoQuery[field] = { ...mongoQuery[field], $lt: queryObj[key] };
+                mongoQuery[field] = { ...mongoQuery[field], $lt: coerce(queryObj[key]) };
             } else if (key.endsWith('_lte')) {
                 const field = key.replace(/_lte$/, '');
-                mongoQuery[field] = { ...mongoQuery[field], $lte: queryObj[key] };
+                mongoQuery[field] = { ...mongoQuery[field], $lte: coerce(queryObj[key]) };
             } else {
                 mongoQuery[key] = queryObj[key];
             }
         }
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@backend/utils/queryEngine.js` around lines 12 - 29, The loop that builds
mongoQuery in backend/utils/queryEngine.js uses req.query string values directly
(e.g., in the _gt/_gte/_lt/_lte branches), causing lexicographic comparisons;
update the logic in that loop (the for...in over queryObj that constructs
mongoQuery) so that when handling comparison suffixes (_gt, _gte, _lt, _lte) you
attempt to coerce the string value to a Number (e.g., parseFloat and check
!isNaN) and use the numeric value in the $gt/$gte/$lt/$lte operators when valid;
otherwise fall back to the original string. Keep the coercion local to those
branches (do not change non-comparison keys) and preserve existing merging
behavior (mongoQuery[field] = { ...mongoQuery[field], $gt: ... } etc.).


this.query = this.query.find(mongoQuery);
Comment on lines +7 to +31

@coderabbitai coderabbitai Bot Feb 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

User-supplied query keys are passed directly into MongoDB filter — potential NoSQL injection vector.

An attacker can craft query parameters like $where=... or __proto__ that flow straight into mongoQuery and then into Model.find(). While Mongoose provides some protections, the else branch on line 27 passes arbitrary keys unvalidated. You should either whitelist allowed field names (e.g., check against the collection's schema model) or at minimum reject keys starting with $.

Proposed safeguard 
         for (const key in queryObj) {
+            // Strip any keys that could be Mongo operators
+            const baseField = key.replace(/_(gt|gte|lt|lte)$/, '');
+            if (baseField.startsWith('$') || baseField.startsWith('__')) continue;
+
             if (key.endsWith('_gt')) {
🧰 Tools
🪛 Biome (2.4.4)

[error] 10-10: This callback passed to forEach() iterable method should not return a value.

(lint/suspicious/useIterableCallbackReturn)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@backend/utils/queryEngine.js` around lines 7 - 31, The filter() method builds
mongoQuery from this.queryString and currently copies keys directly into
mongoQuery before calling this.query.find(mongoQuery), which permits NoSQL
injection; fix by sanitizing/whitelisting keys: either derive an allowedFields
list from the model/schema used by this class (e.g., check against
this.model.schema.paths or an explicit allowedFields array) and only copy keys
that exist there, or at minimum drop any key that begins with '$' or contains
prototype pollution names like '__proto__' or 'constructor' before adding to
mongoQuery; ensure special-suffix handlers (keys ending with _gt/_gte/_lt/_lte)
also respect the same whitelist/sanitization before setting field operators.


return this;
}
Comment on lines +7 to +34

@gemini-code-assist gemini-code-assist Bot Feb 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-high high

The filter method is vulnerable to NoSQL injection. It iterates over the queryString object (derived from req.query) and directly assigns values to the mongoQuery object without validation. If an attacker provides an object containing MongoDB operators (e.g., ?field[$ne]=value), Express's query parser (typically qs) will parse this into a nested object, which is then passed directly to Mongoose's find() method. This allows attackers to bypass intended filters or perform unauthorized data retrieval. You should ensure that all filter values are primitives (strings, numbers, or booleans) or use a sanitization library like mongo-sanitize.

To prevent NoSQL injection, ensure that the value is a primitive.

    filter() {
        const queryObj = { ...this.queryString };
        const excludedFields = ['page', 'sort', 'limit', 'fields'];
        excludedFields.forEach(el => delete queryObj[el]);

        const mongoQuery = {};
        for (const key in queryObj) {
            // Prevent NoSQL injection by ensuring the value is a primitive
            if (typeof queryObj[key] === 'object') continue;

            if (key.endsWith('_gt')) {
                const field = key.replace(/_gt$/, '');
                mongoQuery[field] = { ...mongoQuery[field], $gt: queryObj[key] };
            } else if (key.endsWith('_gte')) {
                const field = key.replace(/_gte$/, '');
                mongoQuery[field] = { ...mongoQuery[field], $gte: queryObj[key] };
            } else if (key.endsWith('_lt')) {
                const field = key.replace(/_lt$/, '');
                mongoQuery[field] = { ...mongoQuery[field], $lt: queryObj[key] };
            } else if (key.endsWith('_lte')) {
                const field = key.replace(/_lte$/, '');
                mongoQuery[field] = { ...mongoQuery[field], $lte: queryObj[key] };
            } else {
                mongoQuery[key] = queryObj[key];
            }
        }
        
        this.query = this.query.find(mongoQuery);
        
        return this;
    }


sort() {
if (this.queryString.sort) {
const sortBy = this.queryString.sort.split(',').map(item => {
const [field, order] = item.split(':');
if (order && (order === '-1' || order.toLowerCase() === 'desc')) {
return `-${field}`;
}
return field;
}).join(' ');

this.query = this.query.sort(sortBy);
} else {
this.query = this.query.sort('-createdAt');
}

return this;
}

paginate() {
const page = parseInt(this.queryString.page, 10) || 1;
const limit = parseInt(this.queryString.limit, 10) || 100;
const skip = (page - 1) * limit;

this.query = this.query.skip(skip).limit(limit);

return this;
}
Comment on lines +54 to +62

@coderabbitai coderabbitai Bot Feb 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

No upper bound on limit — allows unbounded data retrieval.

A client can pass limit=10000000 and dump the entire collection in one request, causing potential memory exhaustion and slow responses. Clamp to a reasonable maximum.

Proposed fix 
     paginate() {
+        const MAX_LIMIT = 500;
         const page = parseInt(this.queryString.page, 10) || 1;
-        const limit = parseInt(this.queryString.limit, 10) || 100;
+        const limit = Math.min(parseInt(this.queryString.limit, 10) || 100, MAX_LIMIT);
         const skip = (page - 1) * limit;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
paginate() {
const page = parseInt(this.queryString.page, 10) || 1;
const limit = parseInt(this.queryString.limit, 10) || 100;
const skip = (page - 1) * limit;
this.query = this.query.skip(skip).limit(limit);
return this;
}
paginate() {
const MAX_LIMIT = 500;
const page = parseInt(this.queryString.page, 10) || 1;
const limit = Math.min(parseInt(this.queryString.limit, 10) || 100, MAX_LIMIT);
const skip = (page - 1) * limit;
this.query = this.query.skip(skip).limit(limit);
return this;
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@backend/utils/queryEngine.js` around lines 54 - 62, In paginate(), prevent
unbounded requests by clamping the parsed limit from this.queryString.limit to a
safe maximum (e.g., MAX_LIMIT) before computing skip; parse the incoming limit
as an integer, ensure it's at least 1, then set limit = Math.min(parsedLimit,
MAX_LIMIT) (or a default if missing/invalid), compute skip and apply this.query
= this.query.skip(skip).limit(limit); add a clear constant like MAX_LIMIT near
paginate() for easy adjustment and reference.

}

module.exports = QueryEngine;
261 changes: 0 additions & 261 deletions diff_output.txt
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion frontend/src/components/CollectionTable.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ export default function CollectionTable({ data, activeCollection, onDelete, onVi
{
id: "actions",
header: "Actions",
size: 80,
size: 120,
enableResizing: false,
enableHiding: false,
cell: (info) => (
Expand Down
4 changes: 2 additions & 2 deletions frontend/src/components/DatabaseSidebar.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ export default function DatabaseSidebar({
/* Sidebar Styles - Scoped */
.db-sidebar {
width: 280px;
background: var(--color-bg-sidebar);
border-right: 1px solid var(--color-border);
background: transparent;
border-right: none;
display: flex;
flex-direction: column;
z-index: 100;
Expand Down
Loading