Skip to content

Benchmarks

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

Benchmarks

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
XAFSecurityBenchmark
XAFSecurityBenchmark
 
 
images
images
 
 
README.md
README.md
 
 

README.md

.NET App Security API Benchmark for EF Core and XPO

About the Project

We built this project to test the performance of Object-Relational Mapping (ORM) libraries used with XAF's Security System. We tested two libraries with BenchmarkDotNet:

You can run the benchmarks on your computer or review our test results below.

Run Benchmarks in Your Environment

You can download the project and run the benchmarks on a system different from the one listed in our Test Results section. You can also modify the data model and test cases: measure memory consumption, include scenarios with BLOBs, add reference or collection properties, and so on.

Once you download the project, follow the steps below to run benchmark tests in your environment:

  1. Download and run the DevExpress Unified Component Installer to add DevExpress.Xpo and other libraries to project references.
  2. Edit the connection string in App.config.
  3. Update the ORM library and target framework versions, if necessary.

Get Support

Contact us if you cannot compile or run any of these demo apps or have questions about our tutorials or supported functionality. Submit questions to the DevExpress Support Center or switch to the Issues tab above. We are here to help.

Benchmark Configuration

Data Model Structure

This project uses the following business objects:

Users and Permissions

Our project creates and loads data objects according to the following data access rules:

  1. A user can access a Contact if their Departments match.
  2. A user can access a Task in two cases: a user from the same department is specified in the AssignedTo field, or such a user exists in the task's Contacts collection.

We use the following code to implement these rules.

In Apps without the Security System

    Expression<Func<Contact, bool>> ContactsFilterPredicate(ICustomPermissionPolicyUser currentUser) =>
        contact => contact.Department == currentUser.Department;

    Expression<Func<DemoTask, bool>> TasksFilterPredicate(ICustomPermissionPolicyUser currentUser) =>
        task => task.Contacts.Any(contact => contact.Department.Users.Any(user => user == currentUser)) ||
        ((Contact)task.AssignedTo).Department == currentUser.Department;

We use these filter predicates to load objects in security-free XPO and EF Core tests. This way we obtain the numbers that we compare to tests with integrated Security System.

In Apps with the Security System

    userRole.AddTypePermission<ContactType>(SecurityOperations.FullObjectAccess, SecurityPermissionState.Deny);
    userRole.AddObjectPermission<ContactType>(SecurityOperations.FullObjectAccess,
      $"[Department].[Users][[{keyPropertyName}] == CurrentUserId()].Exists()", SecurityPermissionState.Allow);

    userRole.AddTypePermission<TaskType>(SecurityOperations.FullObjectAccess, SecurityPermissionState.Deny);
    userRole.AddObjectPermission<TaskType>(SecurityOperations.FullObjectAccess,
      $"[Contacts][[Department].[Users][[{keyPropertyName}] == CurrentUserId()].Exists()]", SecurityPermissionState.Allow);

    if(typeof(TaskType).IsSubclassOf(typeof(DevExpress.Persistent.BaseImpl.Task))
        || typeof(TaskType).IsSubclassOf(typeof(XAFSecurityBenchmark.Models.EFCore.Task))) {
        userRole.AddObjectPermission<TaskType>(SecurityOperations.FullObjectAccess,
          $"[AssignedTo].<Contact>[Department].[Users][[{keyPropertyName}] == CurrentUserId()].Exists()", SecurityPermissionState.Allow);
    }
    else {
        userRole.AddObjectPermission<TaskType>(SecurityOperations.FullObjectAccess,
          "Upcast(AssignedTo, 'XAFSecurityBenchmark.Models.EFCore.Contact', 'Department') == CurrentUserDepartment()", SecurityPermissionState.Allow);
    }

Source: DBUpdaterBase.CreateSecurityObjects

Initial Data

  1. Tests that create new objects start with an empty database. The code cleans the database after every test iteration cycle.
  2. Tests that load collections and modify data use the following prepared dataset:
    • The database updater creates five test users specified by the TestSetConfig.Users array.
    • For every User, we generate 5,000 Contacts (a grand total of 25,000 Contacts in the database). The tests read varying number of contacts on each test iteration (see graphs below). The TestSetConfig.ContactCountPerUserToCreate array specifies the numbers for each test run.
    • For every Contact, we generate Tasks. The TestSetConfig.TasksAssignedToContact and TestSetConfig.TasksLinkedToContact specify the number of Tasks assigned to and linked to the Contact, respectively. The database holds a grand total of 500,000 Tasks.
    • For every Contact, we initialize its associated data such as PhoneNumber, Position and Address.

For more information, see the test object creation logic in the TemporaryTestObjectsHelper class.

Test Results

We ran all benchmarks against .NET 10 and used AnyCPU release builds (include warm-up). The test machine had Windows 11 Enterprise x64, local Microsoft SQL Server Express (64-bit) v16.00.1000, 12th Gen Intel(R) Core(TM) i7-12650H 2.70 GHz/ 32GB RAM / SSD.

Needless to say, lower numbers are better.

Scenario #1. Load Contacts for a specific User

Item Count EF Core 10 (No Security), ms EF Core 10 (Security), ms XPO (No Security), ms XPO (Security), ms
10 2.107 10.984 4.623 10.597
20 2.208 18.388 5.729 12.655
50 2.886 27.030 7.529 17.636
100 3.367 33.105 9.554 20.809
250 6.867 28.685 18.851 31.487
500 11.000 39.837 29.749 58.185
1000 15.628 60.998 51.981 115.524
2500 12.628 155.196 109.171 271.799
5000 24.799 329.727 193.381 551.444

Source: XAFSecurityBenchmark.PerformanceTests.PerformanceTestSet.GetContacts

Scenario #2. Load Tasks for a specific User

Item Count EF Core 10 (No Security), ms EF Core 10 (Security), ms XPO (No Security), ms XPO (Security), ms
10 2.282 7.320 6.328 13.467
20 2.573 8.641 7.631 16.822
50 3.321 13.605 11.727 19.907
100 4.184 15.153 19.287 36.719
250 6.724 26.689 35.859 57.788
500 8.330 26.497 79.187 112.654
1000 25.764 43.401 140.031 211.147
2500 40.466 84.441 276.969 470.176
5000 44.689 180.248 427.088 821.836

Source: XAFSecurityBenchmark.PerformanceTests.PerformanceTestSet.GetTasks

Scenario #3. Create a Contact and its associated data (20 Tasks, PhoneNumbers, Positions, Addresses)

Item Count EF Core 10 (No Security), ms EF Core 10 (Security), ms XPO (No Security), ms XPO (Security), ms
10 23.393 46.824 13.328 18.401
20 35.439 59.385 23.193 25.692
50 59.810 109.783 42.083 36.494
100 109.185 194.541 55.225 70.075
250 169.731 364.025 134.397 154.336
500 277.966 634.819 268.911 316.644
1000 553.470 1255.265 558.591 653.269
2500 1464.907 3238.451 1589.795 1839.196
5000 2851.751 6347.802 3892.196 4421.913

Source: XAFSecurityBenchmark.PerformanceTests.PerformanceTestSet.InsertContact

Scenario #4. Create a Contact without associated data

Item Count EF Core 10 (No Security), ms EF Core 10 (Security), ms XPO (No Security), ms XPO (Security), ms
10 10.489 20.040 9.105 12.102
20 16.356 25.147 14.736 17.248
50 37.036 46.320 26.004 27.697
100 46.298 54.903 39.535 36.060
250 91.034 111.456 78.496 79.359
500 154.090 183.233 150.908 165.202
1000 274.029 331.782 311.693 337.326
2500 637.679 745.848 885.310 938.771
5000 1288.965 1529.645 2152.395 2268.469

Source: XAFSecurityBenchmark.PerformanceTests.PerformanceTestSet.InsertEmptyContact

Scenario #5. Load, update, and save Contacts for a specific User

Item Count EF Core 10 (No Security), ms EF Core 10 (Security), ms XPO (No Security), ms XPO (Security), ms
10 3.635 28.133 7.530 10.323
20 6.141 49.999 11.494 19.589
50 11.985 61.371 20.489 34.017
100 19.058 99.161 32.650 40.921
250 25.876 209.069 54.313 86.007
500 23.046 404.738 88.165 166.444
1000 43.155 828.391 176.571 363.550
2500 98.765 2090.521 484.320 1035.887
5000 215.201 4235.343 999.037 2087.248

Source: XAFSecurityBenchmark.PerformanceTests.PerformanceTestSet.UpdateContacts

Scenario #6. Load, update, and save Tasks for a specific User

Item Count EF Core 10 (No Security), ms EF Core 10 (Security), ms XPO (No Security), ms XPO (Security), ms
10 1.797 26.144 7.918 19.020
20 2.121 38.719 11.345 23.446
50 2.639 59.752 18.821 39.526
100 3.469 63.270 33.037 47.049
250 6.120 122.119 56.530 104.082
500 16.192 247.407 94.784 208.214
1000 20.537 466.231 185.876 457.090
2500 26.693 1158.399 435.882 1056.764
5000 29.882 2339.267 846.801 2096.139

Source: XAFSecurityBenchmark.PerformanceTests.PerformanceTestSet.UpdateTasks