Skip to content

Commit 0bedf89

Browse files
shashank-harnessvcalasanshHenAmarlasdoxgn-harness
authored
[feat]: [PL-25585]: Use Events Framework for NG Ldap Group sync (#36582)
* [feat]: [PL-25585]: Update hashcode Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Update with develop Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Fixed getLdap settings from CG for existing settings. Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Updated endpoints for getting ldap settings from CG Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Fixed Test cases Signed-off-by: Shashank Singh <shashank.singh@harness.io> * fix: [CDS-41642]: Fix NullPointerException in `aggregateStatus` method of GraphRenderer class (#36386) * [CDS-41642]: Fix null pointer exception in aggregateStatus method * [CDS-41642]: Fix codeformat * Address suggestion * [CDS-41642]: Log when stateExecutionInstance status with null value is being aggregated * fix: [CI-4477]: remove prints for gitlab refresh token (#36528) * [fix]: [CDS-39326]: fixing helmchart filter (#36305) * fixing helmchart * fix lint * feat: [PL-27456]: Rename the target to be module and make run analysis per module (#36488) * Rename the target to be module and make run analysis per module * Cleaned up code after code review comments. * feat: [CI-5283]: setting pod timeout for hosted/free users. (#36463) * Setting pod timeout to be 1 hour for hosted free users * Unit test * fix * set active time for direct * move ttl extraction to the helper * remove unused import * removed unused field * Code owner for build-cleaner. (#36536) * fix: [PL-27456]: Include empty deps list in BUILD files (#36455) * fix: [CCM-8659]: Update Anomaly Details Links (#36486) * fix: [CCM-8659]: update anomaly details links * fix: [CCM-8659]: codeformat * fix: [CCM-8659]: unused variables removed * fix: [CCM-8336]: Recommendation doc update (#36411) * fix: [CCM-8336]: recommendation doc update * fix: [CCM-8336]: Recommendation doc update codeformat * feat: [CI-4692]: Git clone step implementation (#34679) * feat: [CI-4692]: Git Clone Step - Initial implementation * feat: [CI-4692]: Git Clone Step - Resolve code review comments * feat: [CI-4692]: Git Clone Step - PR check resolution * feat: [CI-4692]: Git Clone Step - When no cloneDirectory provided, default to /harness/repoName. Do not allow /harness to be used as a git clone step clone directory. * feat: [CI-4692]: Git Clone Step - Maintain drone env variables for all containers except git clone step containers * feat: [CI-4692]: Git Clone Step - add exception for null build environment variables * feat: [CI-4692]: Git Clone Step - clear codebase env variables in GitClone method of plugin setting utils for VM. * feat: [CI-4692]: Git Clone Step - code review comment improvements * feat: [CI-4692]: Git Clone Step - PR check PMD issue fixed * Revert "[CDS-41724]: Adding traits for CD license usage in CG" (#36539) This reverts commit ce53f13. * [feat]: [PL-25585]: sync groups trigger through event * [feat]: [PL-27534]: Ldap test authentication API in NG (cherry picked from commit 8c7daa3) * [feat]: [PL-27534]: Ldap test authentication API in NG (cherry picked from commit 1977354) * [feat]: [PL-25585]: Added delete check and ng group link publish event. Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Added delete check and ng group link publish event. Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Added delete check and ng group link publish event. Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Remove sync on NG Group link Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Fix tests Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Fix style checks Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Fix review comments Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Removed ngLdapGroupSyncEventPublisher rom NG as its not required Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Removed unnecessary comments Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: Removed unnecessary comments Signed-off-by: Shashank Singh <shashank.singh@harness.io> * [feat]: [PL-25585]: add anotation for feature restriction check Signed-off-by: Shashank Singh <shashank.singh@harness.io> Co-authored-by: Vinícius Calasans <vinicius.calasans@harness.io> Co-authored-by: Hen Amar <hen.amar@harness.io> Co-authored-by: Lucas Sales <lucas.sales@harness.io> Co-authored-by: Gaurav Nanda <gaurav.nanda@harness.io> Co-authored-by: Hemanth Mantri <hemanth.mantri@harness.io> Co-authored-by: Trunapushpa <trunapushpa.surkar@harness.io> Co-authored-by: James Ricks <james.ricks@hitachivantara.com> Co-authored-by: Shubham Maheshwari <shubham.maheshwari@harness.io> Co-authored-by: Prateek Barapatre <prateek.barapatre@harness.io>
1 parent f3ad811 commit 0bedf89

30 files changed

Lines changed: 468 additions & 70 deletions

File tree

‎120-ng-manager/modules/ldap/src/main/java/io/harness/ldap/resource/NGLdapResourceImpl.java‎

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717

1818
import software.wings.beans.sso.LdapGroupResponse;
1919
import software.wings.beans.sso.LdapSettings;
20-
import software.wings.beans.sso.LdapSettingsMapper;
2120
import software.wings.beans.sso.LdapTestResponse;
2221

2322
import com.google.inject.Inject;
@@ -37,24 +36,24 @@ public class NGLdapResourceImpl implements NGLdapResource {
3736
@Override
3837
public RestResponse<LdapTestResponse> validateLdapConnectionSettings(
3938
String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings) {
40-
LdapTestResponse ldapTestResponse = ngLdapService.validateLdapConnectionSettings(
41-
accountIdentifier, orgIdentifier, projectIdentifier, LdapSettingsMapper.ldapSettingsDTO(settings));
39+
LdapTestResponse ldapTestResponse =
40+
ngLdapService.validateLdapConnectionSettings(accountIdentifier, orgIdentifier, projectIdentifier, settings);
4241
return new RestResponse<>(ldapTestResponse);
4342
}
4443

4544
@Override
4645
public RestResponse<LdapTestResponse> validateLdapUserSettings(
4746
String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings) {
48-
LdapTestResponse ldapTestResponse = ngLdapService.validateLdapUserSettings(
49-
accountIdentifier, orgIdentifier, projectIdentifier, LdapSettingsMapper.ldapSettingsDTO(settings));
47+
LdapTestResponse ldapTestResponse =
48+
ngLdapService.validateLdapUserSettings(accountIdentifier, orgIdentifier, projectIdentifier, settings);
5049
return new RestResponse<>(ldapTestResponse);
5150
}
5251

5352
@Override
5453
public RestResponse<LdapTestResponse> validateLdapGroupSettings(
5554
String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings) {
56-
LdapTestResponse ldapTestResponse = ngLdapService.validateLdapGroupSettings(
57-
accountIdentifier, orgIdentifier, projectIdentifier, LdapSettingsMapper.ldapSettingsDTO(settings));
55+
LdapTestResponse ldapTestResponse =
56+
ngLdapService.validateLdapGroupSettings(accountIdentifier, orgIdentifier, projectIdentifier, settings);
5857
return new RestResponse<>(ldapTestResponse);
5958
}
6059

‎120-ng-manager/modules/ldap/src/main/java/io/harness/ldap/scheduler/LdapGroupSyncStreamConsumer.java‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
/*
2+
* Copyright 2021 Harness Inc. All rights reserved.
3+
* Use of this source code is governed by the PolyForm Shield 1.0.0 license
4+
* that can be found in the licenses directory at the root of this repository, also available at
5+
* https://polyformproject.org/wp-content/uploads/2020/06/PolyForm-Shield-1.0.0.txt.
6+
*/
7+
8+
package io.harness.ldap.scheduler;
9+
10+
import static io.harness.AuthorizationServiceHeader.NG_MANAGER;
11+
import static io.harness.annotations.dev.HarnessTeam.PL;
12+
import static io.harness.eventsframework.EventsFrameworkConstants.LDAP_GROUP_SYNC;
13+
14+
import io.harness.annotations.dev.OwnedBy;
15+
import io.harness.eventsframework.api.Consumer;
16+
import io.harness.eventsframework.api.EventsFrameworkDownException;
17+
import io.harness.eventsframework.consumer.Message;
18+
import io.harness.eventsframework.ldapgroupsync.ldapgroupsyncdata.LdapGroupSyncDTO;
19+
import io.harness.ldap.service.NGLdapService;
20+
import io.harness.security.SecurityContextBuilder;
21+
import io.harness.security.dto.ServicePrincipal;
22+
23+
import com.google.inject.Inject;
24+
import com.google.inject.Singleton;
25+
import com.google.inject.name.Named;
26+
import com.google.protobuf.InvalidProtocolBufferException;
27+
import java.time.Duration;
28+
import java.util.List;
29+
import java.util.concurrent.TimeUnit;
30+
import lombok.extern.slf4j.Slf4j;
31+
32+
@OwnedBy(PL)
33+
@Slf4j
34+
@Singleton
35+
public class LdapGroupSyncStreamConsumer implements Runnable {
36+
private static final int WAIT_TIME_IN_SECONDS = 10;
37+
private final Consumer redisConsumer;
38+
@Inject NGLdapService ngLdapService;
39+
@Inject
40+
public LdapGroupSyncStreamConsumer(@Named(LDAP_GROUP_SYNC) Consumer redisConsumer) {
41+
this.redisConsumer = redisConsumer;
42+
}
43+
44+
@Override
45+
public void run() {
46+
log.info("EVENT_LDAP_GROUP_SYNC: Started the consumer for ldap group sync stream");
47+
try {
48+
SecurityContextBuilder.setContext(new ServicePrincipal(NG_MANAGER.getServiceId()));
49+
while (!Thread.currentThread().isInterrupted()) {
50+
readEventsFrameworkMessages();
51+
}
52+
} catch (InterruptedException ex) {
53+
SecurityContextBuilder.unsetCompleteContext();
54+
Thread.currentThread().interrupt();
55+
} catch (Exception ex) {
56+
log.error("EVENT_LDAP_GROUP_SYNC: ldap group sync stream consumer unexpectedly stopped", ex);
57+
} finally {
58+
SecurityContextBuilder.unsetCompleteContext();
59+
}
60+
}
61+
62+
private void readEventsFrameworkMessages() throws InterruptedException {
63+
try {
64+
pollAndProcessMessages();
65+
} catch (EventsFrameworkDownException e) {
66+
log.error(
67+
"EVENT_LDAP_GROUP_SYNC: Events framework is down for ldap group sync stream consumer. Retrying again...", e);
68+
TimeUnit.SECONDS.sleep(WAIT_TIME_IN_SECONDS);
69+
}
70+
}
71+
72+
private void pollAndProcessMessages() {
73+
List<Message> messages;
74+
String messageId;
75+
boolean messageProcessed;
76+
messages = redisConsumer.read(Duration.ofSeconds(10));
77+
for (Message message : messages) {
78+
messageId = message.getId();
79+
messageProcessed = handleMessage(message);
80+
if (messageProcessed) {
81+
redisConsumer.acknowledge(messageId);
82+
}
83+
}
84+
}
85+
86+
private boolean handleMessage(Message message) {
87+
try {
88+
processMessage(message);
89+
return true;
90+
} catch (Exception ex) {
91+
// This is not evicted from events framework so that it can be processed
92+
// by other consumer if the error is a runtime error
93+
log.error(
94+
String.format("EVENT_LDAP_GROUP_SYNC: Error occurred in processing message with id %s", message.getId()), ex);
95+
return false;
96+
}
97+
}
98+
99+
private void processMessage(Message message) {
100+
if (message.hasMessage()) {
101+
LdapGroupSyncDTO ldapGroupSyncDTO;
102+
try {
103+
ldapGroupSyncDTO = LdapGroupSyncDTO.parseFrom(message.getMessage().getData());
104+
ngLdapService.syncUserGroupsJob(ldapGroupSyncDTO.getAccountIdentifier(), null, null);
105+
} catch (InvalidProtocolBufferException e) {
106+
log.error("EVENT_LDAP_GROUP_SYNC: Exception in unpacking ldapGroupSyncDTO for key {}", message.getId(), e);
107+
throw new IllegalStateException(e);
108+
}
109+
}
110+
}
111+
}

‎120-ng-manager/modules/ldap/src/main/java/io/harness/ldap/service/NGLdapService.java‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,22 +10,21 @@
1010
import io.harness.annotations.dev.HarnessTeam;
1111
import io.harness.annotations.dev.OwnedBy;
1212

13-
import software.wings.beans.dto.LdapSettings;
1413
import software.wings.beans.sso.LdapGroupResponse;
14+
import software.wings.beans.sso.LdapSettings;
1515
import software.wings.beans.sso.LdapTestResponse;
1616

1717
import java.util.Collection;
18-
import javax.validation.Valid;
1918
import javax.validation.constraints.NotNull;
2019

2120
@OwnedBy(HarnessTeam.PL)
2221
public interface NGLdapService {
2322
LdapTestResponse validateLdapConnectionSettings(
24-
@NotNull String accountIdentifier, String orgIdentifier, String projectIdentifier, @Valid LdapSettings settings);
23+
@NotNull String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings);
2524
LdapTestResponse validateLdapUserSettings(
26-
@NotNull String accountIdentifier, String orgIdentifier, String projectIdentifier, @Valid LdapSettings settings);
25+
@NotNull String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings);
2726
LdapTestResponse validateLdapGroupSettings(
28-
@NotNull String accountIdentifier, String orgIdentifier, String projectIdentifier, @Valid LdapSettings settings);
27+
@NotNull String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings);
2928
Collection<LdapGroupResponse> searchLdapGroupsByName(@NotNull String accountIdentifier, String orgIdentifier,
3029
String projectIdentifier, @NotNull String ldapId, @NotNull String name);
3130
void syncUserGroupsJob(@NotNull String accountIdentifier, String orgIdentifier, String projectIdentifier);

‎120-ng-manager/modules/ldap/src/main/java/io/harness/ldap/service/impl/NGLdapServiceImpl.java‎

Lines changed: 36 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,13 @@
4040
import io.harness.ng.core.api.UserGroupService;
4141
import io.harness.ng.core.user.entities.UserGroup;
4242
import io.harness.rest.RestResponse;
43+
import io.harness.security.encryption.EncryptedDataDetail;
4344
import io.harness.service.DelegateGrpcClientWrapper;
4445

4546
import software.wings.beans.TaskType;
4647
import software.wings.beans.dto.LdapSettings;
4748
import software.wings.beans.sso.LdapGroupResponse;
49+
import software.wings.beans.sso.LdapSettingsMapper;
4850
import software.wings.beans.sso.LdapTestResponse;
4951
import software.wings.service.impl.ldap.LdapDelegateException;
5052

@@ -73,8 +75,8 @@ public class NGLdapServiceImpl implements NGLdapService {
7375

7476
@Override
7577
public LdapTestResponse validateLdapConnectionSettings(String accountIdentifier, String orgIdentifier,
76-
String projectIdentifier, software.wings.beans.dto.LdapSettings settings) {
77-
NGLdapDelegateTaskParameters parameters = NGLdapDelegateTaskParameters.builder().ldapSettings(settings).build();
78+
String projectIdentifier, software.wings.beans.sso.LdapSettings settings) {
79+
NGLdapDelegateTaskParameters parameters = getNgLdapDelegateTaskParameters(accountIdentifier, settings);
7880

7981
DelegateResponseData delegateResponseData = getDelegateResponseData(
8082
accountIdentifier, orgIdentifier, projectIdentifier, parameters, NG_LDAP_TEST_CONN_SETTINGS);
@@ -86,9 +88,9 @@ public LdapTestResponse validateLdapConnectionSettings(String accountIdentifier,
8688
}
8789

8890
@Override
89-
public LdapTestResponse validateLdapUserSettings(
90-
String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings) {
91-
NGLdapDelegateTaskParameters parameters = NGLdapDelegateTaskParameters.builder().ldapSettings(settings).build();
91+
public LdapTestResponse validateLdapUserSettings(String accountIdentifier, String orgIdentifier,
92+
String projectIdentifier, software.wings.beans.sso.LdapSettings settings) {
93+
NGLdapDelegateTaskParameters parameters = getNgLdapDelegateTaskParameters(accountIdentifier, settings);
9294

9395
DelegateResponseData delegateResponseData = getDelegateResponseData(
9496
accountIdentifier, orgIdentifier, projectIdentifier, parameters, NG_LDAP_TEST_USER_SETTINGS);
@@ -100,9 +102,9 @@ public LdapTestResponse validateLdapUserSettings(
100102
}
101103

102104
@Override
103-
public LdapTestResponse validateLdapGroupSettings(
104-
String accountIdentifier, String orgIdentifier, String projectIdentifier, LdapSettings settings) {
105-
NGLdapDelegateTaskParameters parameters = NGLdapDelegateTaskParameters.builder().ldapSettings(settings).build();
105+
public LdapTestResponse validateLdapGroupSettings(String accountIdentifier, String orgIdentifier,
106+
String projectIdentifier, software.wings.beans.sso.LdapSettings settings) {
107+
NGLdapDelegateTaskParameters parameters = getNgLdapDelegateTaskParameters(accountIdentifier, settings);
106108

107109
DelegateResponseData delegateResponseData = getDelegateResponseData(
108110
accountIdentifier, orgIdentifier, projectIdentifier, parameters, NG_LDAP_TEST_GROUP_SETTINGS);
@@ -138,6 +140,7 @@ public Collection<LdapGroupResponse> searchLdapGroupsByName(
138140

139141
@Override
140142
public void syncUserGroupsJob(String accountIdentifier, String orgIdentifier, String projectIdentifier) {
143+
// TODO: Add more logs
141144
LdapSettingsWithEncryptedDataDetail settingsWithEncryptedDataDetail =
142145
getLdapSettingsWithEncryptedDataInternal(accountIdentifier);
143146

@@ -167,9 +170,33 @@ public void syncUserGroupsJob(String accountIdentifier, String orgIdentifier, St
167170
userGroupsToLdapGroupMap, settingsWithEncryptedDataDetail.getLdapSettings().getUuid(), accountIdentifier);
168171
}
169172

173+
private NGLdapDelegateTaskParameters getNgLdapDelegateTaskParameters(
174+
String accountIdentifier, software.wings.beans.sso.LdapSettings settings) {
175+
LdapSettingsWithEncryptedDataDetail settingsWithEncryptedDataDetail =
176+
getLdapSettingsWithEncryptedDataInternal(accountIdentifier, settings);
177+
EncryptedDataDetail encryptedDataDetail = settingsWithEncryptedDataDetail.getEncryptedDataDetail();
178+
return NGLdapDelegateTaskParameters.builder()
179+
.ldapSettings(settingsWithEncryptedDataDetail.getLdapSettings())
180+
.encryptedDataDetail(encryptedDataDetail)
181+
.build();
182+
}
183+
184+
private LdapSettingsWithEncryptedDataDetail getLdapSettingsWithEncryptedDataInternal(
185+
String accountIdentifier, software.wings.beans.sso.LdapSettings ldapSettings) {
186+
Call<RestResponse<LdapSettingsWithEncryptedDataDetail>> settingsWithEncryptedDataDetails =
187+
managerClient.getLdapSettingsUsingAccountIdAndLdapSettings(
188+
accountIdentifier, LdapSettingsMapper.ldapSettingsDTO(ldapSettings));
189+
if (null == settingsWithEncryptedDataDetails) {
190+
log.warn(
191+
"Failed to get ldap settings with encrypted data detail from manager for account: {}", accountIdentifier);
192+
throw new InvalidRequestException("Failed to get LDAPSettings with encrypted data detail for the request");
193+
}
194+
return getResponse(settingsWithEncryptedDataDetails);
195+
}
196+
170197
private LdapSettingsWithEncryptedDataDetail getLdapSettingsWithEncryptedDataInternal(String accountIdentifier) {
171198
Call<RestResponse<LdapSettingsWithEncryptedDataDetail>> settingsWithEncryptedDataDetails =
172-
managerClient.getLdapSettingsWithEncryptedDataDetails(accountIdentifier);
199+
managerClient.getLdapSettingsUsingAccountId(accountIdentifier);
173200
if (null == settingsWithEncryptedDataDetails) {
174201
log.warn(
175202
"Failed to get ldap settings with encrypted data detail from manager for account: {}", accountIdentifier);

‎120-ng-manager/modules/ldap/src/test/java/io/harness/ldap/resource/NGLdapResourceImplTest.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ public void testSearchLdapGroups() throws IOException {
7070
int totalMembers = 4;
7171
final String groupQueryStr = "testGroupName";
7272
Call<RestResponse<LdapSettingsWithEncryptedDataDetail>> request = mock(Call.class);
73-
doReturn(request).when(managerClient).getLdapSettingsWithEncryptedDataDetails(ACCOUNT_ID);
73+
doReturn(request).when(managerClient).getLdapSettingsUsingAccountId(ACCOUNT_ID);
7474
RestResponse<LdapSettingsWithEncryptedDataDetail> mockResponse =
7575
new RestResponse<>(ldapSettingsWithEncryptedDataDetail);
7676
doReturn(Response.success(mockResponse)).when(request).execute();

0 commit comments

Comments
 (0)