Skip to content

Prometheus exporter enhancement #4438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DaanHoogland merged 17 commits into from
Sep 30, 2022
Merged

Prometheus exporter enhancement #4438

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1984b8a
Export count of total/up/down hosts by tags
Sep 15, 2020
d75b66f
Export count of vms by state and host tag.
Sep 15, 2020
1b24280
Add host tags to host cpu/cores/memory usage in Prometheus exporter
Sep 17, 2020
ab04936
Cloudstack Prometheus exporter: Add allocated capacity group by host …
Sep 17, 2020
230d3d7
Show count of Active domains on grafana.
Sep 21, 2020
1687b28
Show count of Active accounts and vms by size on grafana
Sep 24, 2020
6a6e18b
Fixed HostTagsDao import issue and removed unnecessary fields.
Oct 30, 2020
7a897d3
Change variable names with regards to DaanHoogland comments.
Aug 9, 2021
d41b99c
Moved to commons.lang3 as a default string lib.
Sep 23, 2021
cbd780d
Fixed spaces.
Sep 23, 2021
fed9b6a
Updated pr to address @GutoVeronezi change requests.
Sep 23, 2021
633ebac
Fixed camel case issue.
Sep 27, 2021
bfd8926
Removed redundant lines of the Prometheus outputs.
Dec 13, 2021
76fbc2a
Use prepared statement to query database for a number of VM who use a…
Dec 15, 2021
0bcc6bc
Rename gethostTags to getHostTags.
Mar 24, 2022
51571e4
Remove unwanted files.
Mar 24, 2022
6aab64b
Extract repeated codes to new methods.
Mar 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions engine/schema/src/main/java/com/cloud/capacity/dao/CapacityDao.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import com.cloud.capacity.CapacityVO;
import com.cloud.capacity.dao.CapacityDaoImpl.SummedCapacity;
import com.cloud.utils.Pair;
import com.cloud.utils.Ternary;
import com.cloud.utils.db.GenericDao;

public interface CapacityDao extends GenericDao<CapacityVO, Long> {
Expand All @@ -39,6 +40,8 @@ public interface CapacityDao extends GenericDao<CapacityVO, Long> {

Pair<List<Long>, Map<Long, Double>> orderClustersByAggregateCapacity(long id, long vmId, short capacityType, boolean isZone);

Ternary<Long, Long, Long> findCapacityByZoneAndHostTag(Long zoneId, String hostTag);

List<SummedCapacity> findCapacityBy(Integer capacityType, Long zoneId, Long podId, Long clusterId);

List<Long> listPodsByHostCapacities(long zoneId, int requiredCpu, long requiredRam, short capacityType);
Expand Down
52 changes: 47 additions & 5 deletions engine/schema/src/main/java/com/cloud/capacity/dao/CapacityDaoImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import javax.inject.Inject;

import org.apache.log4j.Logger;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Component;

import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao;
Expand All @@ -37,6 +38,7 @@
import com.cloud.dc.ClusterDetailsDao;
import com.cloud.storage.Storage;
import com.cloud.utils.Pair;
import com.cloud.utils.Ternary;
import com.cloud.utils.db.GenericDaoBase;
import com.cloud.utils.db.GenericSearchBuilder;
import com.cloud.utils.db.JoinBuilder.JoinType;
Expand Down Expand Up @@ -203,11 +205,15 @@ public class CapacityDaoImpl extends GenericDaoBase<CapacityVO, Long> implements
"FROM (SELECT vi.data_center_id, (CASE WHEN ISNULL(service_offering.cpu) THEN custom_cpu.value ELSE service_offering.cpu end) AS cpu, " +
"(CASE WHEN ISNULL(service_offering.speed) THEN custom_speed.value ELSE service_offering.speed end) AS speed, " +
"(CASE WHEN ISNULL(service_offering.ram_size) THEN custom_ram_size.value ELSE service_offering.ram_size end) AS ram_size " +
"FROM (((vm_instance vi LEFT JOIN service_offering ON(((vi.service_offering_id = service_offering.id))) " +
"LEFT JOIN user_vm_details custom_cpu ON(((custom_cpu.vm_id = vi.id) AND (custom_cpu.name = 'CpuNumber')))) " +
"LEFT JOIN user_vm_details custom_speed ON(((custom_speed.vm_id = vi.id) AND (custom_speed.name = 'CpuSpeed')))) " +
"LEFT JOIN user_vm_details custom_ram_size ON(((custom_ram_size.vm_id = vi.id) AND (custom_ram_size.name = 'memory')))) " +
"WHERE ISNULL(vi.removed) AND vi.state NOT IN ('Destroyed', 'Error', 'Expunging')";
"FROM vm_instance vi LEFT JOIN service_offering ON(((vi.service_offering_id = service_offering.id))) " +
Copy link
Copy Markdown
Member

@yadvr yadvr Apr 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@soreana all your changes LGTM, I've one question - do you see this change in LIST_ALLOCATED_CAPACITY_GROUP_BY_CAPACITY_AND_ZONE causing any potential regression in the users of findCapacityBy() method, and should we do any kind of testing for this cc @nvazquez.

However, I do see use of WHERE_STATE_IS_NOT_DESTRUCTIVE which you append to the sql string, but whether it has impact on the overall sql query?

Copy link
Copy Markdown
Member

@yadvr yadvr Jun 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @soreana cc @ravening we're waiting for your confirmation and then we can merge your PR. Any update/comment ?

Copy link
Copy Markdown
Member Author

@soreana soreana Jun 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rohityadavcloud Sorry I was away and missed this comment. What do you mean by regression?
I guess that you mean the findCapacityBy() returns different values for different cases in a new version, but it returns the same in an old version. Am I correctly understanding your comment?

Copy link
Copy Markdown
Member

@yadvr yadvr Jun 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @soreana yes, my concern is there are code changes in findCapacityBy() method and my concern is if that could cause regression to consumers of this method (outside of your use-case)? Could you check if my concerns are valid, if not I'm happy to conclude the review the merge the PR.

Copy link
Copy Markdown
Member

@yadvr yadvr Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @soreana could you confirm if my concerns are valid, or we can go ahead and merge the PR?

Copy link
Copy Markdown
Member

@yadvr yadvr Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @DaanHoogland

Copy link
Copy Markdown
Contributor

@DaanHoogland DaanHoogland Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@soreana I don´t see any ansswer yet. Can you explain the method, how it is changed and how this impacts other uses please?

Copy link
Copy Markdown
Member Author

@soreana soreana Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DaanHoogland @rohityadavcloud Sorry for late respond, I've lost track of this issue. No it doesn't change affect the findCapacityBy() return value. The where clause appended to the SQL query later in the code.

https://github.com/apache/cloudstack/pull/4438/files#diff-dfeaef8c3e4c5eb658e6d479055c7ffdc407a9e959de834b76e881e7b140faf5R474

"LEFT JOIN user_vm_details custom_cpu ON(((custom_cpu.vm_id = vi.id) AND (custom_cpu.name = 'CpuNumber'))) " +
"LEFT JOIN user_vm_details custom_speed ON(((custom_speed.vm_id = vi.id) AND (custom_speed.name = 'CpuSpeed'))) " +
"LEFT JOIN user_vm_details custom_ram_size ON(((custom_ram_size.vm_id = vi.id) AND (custom_ram_size.name = 'memory'))) ";

private static final String WHERE_STATE_IS_NOT_DESTRUCTIVE =
"WHERE ISNULL(vi.removed) AND vi.state NOT IN ('Destroyed', 'Error', 'Expunging')";

private static final String LEFT_JOIN_VM_TEMPLATE = "LEFT JOIN vm_template ON vm_template.id = vi.vm_template_id ";

public CapacityDaoImpl() {
_hostIdTypeSearch = createSearchBuilder();
Expand Down Expand Up @@ -422,6 +428,41 @@ public List<SummedCapacity> listCapacitiesGroupedByLevelAndType(Integer capacity

}

@Override
public Ternary<Long, Long, Long> findCapacityByZoneAndHostTag(Long zoneId, String hostTag) {
TransactionLegacy txn = TransactionLegacy.currentTxn();
PreparedStatement pstmt;

StringBuilder allocatedSql = new StringBuilder(LIST_ALLOCATED_CAPACITY_GROUP_BY_CAPACITY_AND_ZONE);
if (StringUtils.isNotEmpty(hostTag)) {
allocatedSql.append(LEFT_JOIN_VM_TEMPLATE);
}
allocatedSql.append(WHERE_STATE_IS_NOT_DESTRUCTIVE);
if (zoneId != null) {
allocatedSql.append(" AND vi.data_center_id = ?");
}
if (StringUtils.isNotEmpty(hostTag)) {
allocatedSql.append(" AND (vm_template.template_tag = '").append(hostTag).append("'");
allocatedSql.append(" OR service_offering.host_tag = '").append(hostTag).append("')");
}
allocatedSql.append(" ) AS v GROUP BY v.data_center_id");

try {
// add allocated capacity of zone in result
pstmt = txn.prepareAutoCloseStatement(allocatedSql.toString());
if (zoneId != null) {
pstmt.setLong(1, zoneId);
}
ResultSet rs = pstmt.executeQuery();
if (rs.next()) {
return new Ternary<>(rs.getLong(2), rs.getLong(3), rs.getLong(4)); // cpu cores, cpu, memory
}
return new Ternary<>(0L, 0L, 0L);
} catch (SQLException e) {
throw new CloudRuntimeException("DB Exception on: " + allocatedSql, e);
}
}

@Override
public List<SummedCapacity> findCapacityBy(Integer capacityType, Long zoneId, Long podId, Long clusterId) {

Expand All @@ -430,6 +471,7 @@ public List<SummedCapacity> findCapacityBy(Integer capacityType, Long zoneId, Lo
List<SummedCapacity> results = new ArrayList<SummedCapacity>();

StringBuilder allocatedSql = new StringBuilder(LIST_ALLOCATED_CAPACITY_GROUP_BY_CAPACITY_AND_ZONE);
allocatedSql.append(WHERE_STATE_IS_NOT_DESTRUCTIVE);

HashMap<Long, Long> sumCpuCore = new HashMap<Long, Long>();
HashMap<Long, Long> sumCpu = new HashMap<Long, Long>();
Expand Down
1 change: 1 addition & 0 deletions engine/schema/src/main/java/com/cloud/user/dao/AccountDao.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,5 @@ public interface AccountDao extends GenericDao<AccountVO, Long> {
*/
long getDomainIdForGivenAccountId(long id);

int getActiveDomains();
}
16 changes: 15 additions & 1 deletion engine/schema/src/main/java/com/cloud/user/dao/AccountDaoImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import com.cloud.utils.db.GenericSearchBuilder;
import com.cloud.utils.db.SearchBuilder;
import com.cloud.utils.db.SearchCriteria;
import com.cloud.utils.db.SearchCriteria.Func;
import com.cloud.utils.db.SearchCriteria.Op;
import org.apache.commons.lang3.StringUtils;
import com.cloud.utils.db.TransactionLegacy;
Expand All @@ -54,6 +55,7 @@ public class AccountDaoImpl extends GenericDaoBase<AccountVO, Long> implements A
protected final SearchBuilder<AccountVO> NonProjectAccountSearch;
protected final SearchBuilder<AccountVO> AccountByRoleSearch;
protected final GenericSearchBuilder<AccountVO, Long> AccountIdsSearch;
protected final GenericSearchBuilder<AccountVO, Long> ActiveDomainCount;

public AccountDaoImpl() {
AllFieldsSearch = createSearchBuilder();
Expand Down Expand Up @@ -101,6 +103,13 @@ public AccountDaoImpl() {
AccountByRoleSearch = createSearchBuilder();
AccountByRoleSearch.and("roleId", AccountByRoleSearch.entity().getRoleId(), SearchCriteria.Op.EQ);
AccountByRoleSearch.done();

ActiveDomainCount = createSearchBuilder(Long.class);
ActiveDomainCount.select(null, Func.COUNT, null);
ActiveDomainCount.and("domain", ActiveDomainCount.entity().getDomainId(), SearchCriteria.Op.EQ);
ActiveDomainCount.and("state", ActiveDomainCount.entity().getState(), SearchCriteria.Op.EQ);
ActiveDomainCount.groupBy(ActiveDomainCount.entity().getDomainId());
ActiveDomainCount.done();
}

@Override
Expand Down Expand Up @@ -318,5 +327,10 @@ public long getDomainIdForGivenAccountId(long id) {
}
}


@Override
public int getActiveDomains() {
SearchCriteria<Long> sc = ActiveDomainCount.create();
sc.setParameters("state", "enabled");
return customSearch(sc, null).size();
}
}
5 changes: 5 additions & 0 deletions engine/schema/src/main/java/com/cloud/vm/dao/UserVmDao.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import java.util.Set;

import com.cloud.utils.Pair;
import com.cloud.utils.Ternary;
import com.cloud.utils.db.GenericDao;
import com.cloud.vm.UserVmVO;
import com.cloud.vm.VirtualMachine;
Expand Down Expand Up @@ -92,4 +93,8 @@ public interface UserVmDao extends GenericDao<UserVmVO, Long> {
List<UserVmVO> listByIsoId(Long isoId);

List<Pair<Pair<String, VirtualMachine.Type>, Pair<Long, String>>> getVmsDetailByNames(Set<String> vmNames, String detail);

List<Ternary<Integer, Integer, Integer>> countVmsBySize(long dcId, int limit);

int getActiveAccounts(final long dcId);
}
42 changes: 42 additions & 0 deletions engine/schema/src/main/java/com/cloud/vm/dao/UserVmDaoImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import com.cloud.tags.dao.ResourceTagDao;
import com.cloud.user.Account;
import com.cloud.utils.Pair;
import com.cloud.utils.Ternary;
import com.cloud.utils.db.Attribute;
import com.cloud.utils.db.GenericDaoBase;
import com.cloud.utils.db.GenericSearchBuilder;
Expand Down Expand Up @@ -75,6 +76,7 @@ public class UserVmDaoImpl extends GenericDaoBase<UserVmVO, Long> implements Use
protected SearchBuilder<UserVmVO> AccountDataCenterVirtualSearch;
protected GenericSearchBuilder<UserVmVO, Long> CountByAccountPod;
protected GenericSearchBuilder<UserVmVO, Long> CountByAccount;
protected GenericSearchBuilder<UserVmVO, Long> CountActiveAccount;
protected GenericSearchBuilder<UserVmVO, Long> PodsHavingVmsForAccount;

protected SearchBuilder<UserVmVO> UserVmSearch;
Expand Down Expand Up @@ -192,6 +194,15 @@ void init() {
CountByAccount.and("displayVm", CountByAccount.entity().isDisplayVm(), SearchCriteria.Op.EQ);
CountByAccount.done();

CountActiveAccount = createSearchBuilder(Long.class);
CountActiveAccount.select(null, Func.COUNT, null);
CountActiveAccount.and("account", CountActiveAccount.entity().getAccountId(), SearchCriteria.Op.EQ);
CountActiveAccount.and("type", CountActiveAccount.entity().getType(), SearchCriteria.Op.EQ);
CountActiveAccount.and("dataCenterId", CountActiveAccount.entity().getDataCenterId(), SearchCriteria.Op.EQ);
CountActiveAccount.and("state", CountActiveAccount.entity().getState(), SearchCriteria.Op.NIN);
CountActiveAccount.groupBy(CountActiveAccount.entity().getAccountId());
CountActiveAccount.done();

SearchBuilder<NicVO> nicSearch = _nicDao.createSearchBuilder();
nicSearch.and("networkId", nicSearch.entity().getNetworkId(), SearchCriteria.Op.EQ);
nicSearch.and("ip4Address", nicSearch.entity().getIPv4Address(), SearchCriteria.Op.NNULL);
Expand Down Expand Up @@ -721,4 +732,35 @@ public List<Pair<Pair<String, VirtualMachine.Type>, Pair<Long, String>>> getVmsD

return vmsDetailByNames;
}

@Override
public List<Ternary<Integer, Integer, Integer>> countVmsBySize(long dcId, int limit) {
TransactionLegacy txn = TransactionLegacy.currentTxn();
String sql = "SELECT cpu,ram_size,count(1) AS count FROM (SELECT * FROM user_vm_view WHERE data_center_id = ? AND state NOT IN ('Destroyed', 'Error', 'Expunging') GROUP BY id) AS uvv GROUP BY cpu,ram_size ORDER BY count DESC ";
if (limit >= 0)
sql = sql + "limit " + limit;
PreparedStatement pstmt = null;
List<Ternary<Integer, Integer, Integer>> result = new ArrayList<>();
try {
pstmt = txn.prepareAutoCloseStatement(sql);
pstmt.setLong(1, dcId);
ResultSet rs = pstmt.executeQuery();
while (rs.next()) {
result.add(new Ternary<Integer, Integer, Integer>(rs.getInt(1), rs.getInt(2), rs.getInt(3)));
}
} catch (Exception e) {
s_logger.warn("Error counting vms by size for dcId= " + dcId, e);
}
return result;
}

@Override
public int getActiveAccounts(final long dcId) {
SearchCriteria<Long> sc = CountActiveAccount.create();
sc.setParameters("type", VirtualMachine.Type.User);
sc.setParameters("state", State.Destroyed, State.Error, State.Expunging, State.Stopped);
sc.setParameters("dataCenterId", dcId);

return customSearch(sc, null).size();
}
}
2 changes: 2 additions & 0 deletions engine/schema/src/main/java/com/cloud/vm/dao/VMInstanceDao.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,8 @@ public interface VMInstanceDao extends GenericDao<VMInstanceVO, Long>, StateDao<

Long countByZoneAndState(long zoneId, State state);

Long countByZoneAndStateAndHostTag(long dcId, State state, String hostTag);

List<VMInstanceVO> listNonRemovedVmsByTypeAndNetwork(long networkId, VirtualMachine.Type... types);

/**
Expand Down
25 changes: 25 additions & 0 deletions engine/schema/src/main/java/com/cloud/vm/dao/VMInstanceDaoImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,9 @@ public class VMInstanceDaoImpl extends GenericDaoBase<VMInstanceVO, Long> implem

private static final String UPDATE_SYSTEM_VM_TEMPLATE_ID_FOR_HYPERVISOR = "UPDATE `cloud`.`vm_instance` SET vm_template_id = ? WHERE type <> 'User' AND hypervisor_type = ? AND removed is NULL";

private static final String COUNT_VMS_BY_ZONE_AND_STATE_AND_HOST_TAG = "SELECT COUNT(1) FROM vm_instance vi JOIN service_offering so ON vi.service_offering_id=so.id " +
"JOIN vm_template vt ON vi.vm_template_id = vt.id WHERE vi.data_center_id = ? AND vi.state = ? AND vi.removed IS NULL AND (so.host_tag = ? OR vt.template_tag = ?)";

@Inject
protected HostDao _hostDao;

Expand Down Expand Up @@ -807,6 +810,28 @@ public Long countByZoneAndState(long zoneId, State state) {
return customSearch(sc, null).get(0);
}

@Override
public Long countByZoneAndStateAndHostTag(long dcId, State state, String hostTag) {
TransactionLegacy txn = TransactionLegacy.currentTxn();
PreparedStatement pstmt = null;
try {
pstmt = txn.prepareAutoCloseStatement(COUNT_VMS_BY_ZONE_AND_STATE_AND_HOST_TAG);

pstmt.setLong(1, dcId);
pstmt.setString(2, String.valueOf(state));
pstmt.setString(3, hostTag);
pstmt.setString(4, hostTag);

ResultSet rs = pstmt.executeQuery();
if (rs.next()) {
return rs.getLong(1);
}
} catch (Exception e) {
s_logger.warn(String.format("Error counting vms by host tag for dcId= %s, hostTag= %s", dcId, hostTag), e);
}
return 0L;
}

@Override
public List<VMInstanceVO> listNonRemovedVmsByTypeAndNetwork(long networkId, VirtualMachine.Type... types) {
if (NetworkTypeSearch == null) {
Expand Down
Loading