performance - 如何提高 apache ignite 中的查询性能

Question

我已经设置了一个由两台服务器和一个客户端组成的 ignite 集群。集群配置如下:

IgniteConfiguration icfg = new IgniteConfiguration();

icfg.setIncludeEventTypes(EventType.EVT_TASK_STARTED,EventType.EVT_TASK_FINISHED,EventType.EVT_TASK_FAILED);

icfg.setMetricsUpdateFrequency(-1);

Ignite ignite = Ignition.start(icfg);

我已禁用指标更新频率，仅启用了 Ignite 文档的性能提示部分中建议的一些事件通知。

我已经在集群中创建了一个具有以下配置的缓存:

CacheConfiguration<Integer,DataPOJO> cacheConfiguration = new CacheConfiguration<>();

cacheConfiguration.setStatisticsEnabled(false);

cacheConfiguration.setName("testCache");

我正在从具有 100 000 个条目的客户端加载缓存。

使用的POJO类代码如下:

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.sql.Date;

import org.apache.ignite.binary.BinaryObjectException;
import org.apache.ignite.binary.BinaryReader;
import org.apache.ignite.binary.BinaryWriter;
import org.apache.ignite.binary.Binarylizable;
import org.apache.ignite.cache.query.annotations.QuerySqlField;

public class DataPOJO implements Externalizable,Binarylizable{

    private static final long serialVersionUID = 1L;

    @QuerySqlField(index = true)
    private int primaryKey;

    @QuerySqlField
    private int foreignKey1;

    @QuerySqlField
    private int foreignKey2;

    @QuerySqlField
    private String stringField1;

    @QuerySqlField
    private String stringField2;

    @QuerySqlField
    private String stringField3;

    @QuerySqlField
    private Date dateField;

    @QuerySqlField(index = true)
    private String stringField4;

    public int getPrimaryKey() {
        return primaryKey;
    }

    public void setPrimaryKey(int primaryKey) {
        this.primaryKey = primaryKey;
    }

    public int getForeignKey1() {
        return foreignKey1;
    }

    public void setForeignKey1(int foreignKey1) {
        this.foreignKey1 = foreignKey1;
    }

    public int getForeignKey2() {
        return foreignKey2;
    }

    public void setForeignKey2(int foreignKey2) {
        this.foreignKey2 = foreignKey2;
    }

    public String getStringField1() {
        return stringField1;
    }

    public void setStringField1(String stringField1) {
        this.stringField1 = stringField1;
    }

    public String getStringField2() {
        return stringField2;
    }

    public void setStringField2(String stringField2) {
        this.stringField2 = stringField2;
    }

    public String getStringField3() {
        return stringField3;
    }

    public void setStringField3(String stringField3) {
        this.stringField3 = stringField3;
    }

    public Date getDateField() {
        return dateField;
    }

    public void setDateField(Date dateField) {
        this.dateField = dateField;
    }

    public String getStringField4() {
        return stringField4;
    }

    public void setStringField4(String stringField4) {
        this.stringField4 = stringField4;
    }

    @Override
    public String toString() {
        return "DataPOJO [primaryKey=" + primaryKey + ", foreignKey1=" + foreignKey1 + ", foreignKey2=" + foreignKey2
                + ", stringField1=" + stringField1 + ", stringField2=" + stringField2 + ", stringField3=" + stringField3
                + ", dateField=" + dateField + ", stringField4=" + stringField4 + "]";
    }

    @Override
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        primaryKey = in.readInt();
        foreignKey1 = in.readInt();
        foreignKey2 = in.readInt();
        stringField1 = (String) in.readObject();
        stringField2 = (String) in.readObject();
        stringField3 = (String) in.readObject();
        stringField4 = (String) in.readObject();
        dateField = (Date) in.readObject();
    }

    @Override
    public void writeExternal(ObjectOutput out) throws IOException {
        out.writeInt(primaryKey);
        out.writeInt(foreignKey1);
        out.writeInt(foreignKey2);
        out.writeObject(stringField1);
        out.writeObject(stringField2);
        out.writeObject(stringField3);
        out.writeObject(stringField4);
        out.writeObject(dateField);
    }

    @Override
    public void writeBinary(BinaryWriter writer) throws BinaryObjectException {
        writer.writeInt("primarykey",primaryKey);
        writer.writeInt("foreignkey1",foreignKey1);
        writer.writeInt("foreignkey2",foreignKey2);
        writer.writeString("stringField1",stringField1);
        writer.writeString("stringField2",stringField2);
        writer.writeString("stringField3",stringField3);
        writer.writeString("stringField4",stringField4);
        writer.writeDate("dateField",new java.util.Date(dateField.getTime()));
    }

    @Override
    public void readBinary(BinaryReader reader) throws BinaryObjectException {
        primaryKey = reader.readInt("primarykey");
        foreignKey1 = reader.readInt("foreignkey1");
        foreignKey2 = reader.readInt("foreignkey2");
        stringField1 = reader.readString("stringField1");
        stringField2 = reader.readString("stringField2");
        stringField3 = reader.readString("stringField3");
        stringField4 = reader.readString("stringField4");
        dateField = new Date(reader.readDate("dateField").getTime());
    }

}

我不想使用 java.io.Serializable，因为它会影响性能。我在两个字段上创建了索引，即“primaryKey”和“stringField4”。

字段 stringField4 将是每个条目的字段 primaryKey 的字符串表示。如果 primaryKey = 1，则 stringField4 = "1"。

primaryKey 也是缓存中任何条目的键。

我从客户端执行的查询是根据字段 stringField4 的 10,000 个值从缓存中获取 10,000 个条目。查询如下:

IgniteCache<Integer,DataPOJO> testCache= ignite.getOrCreateCache("testCache");

SqlQuery<Integer,DataPOJO> query = new SqlQuery<>(DataPOJO.class,"stringField4 = ?");

long startTime,totalTimeElapsed = 0;

QueryCursor<Entry<Integer,DataPOJO>> cursor;

for(int i=1;i<=10000;i++){

    startTime = System.nanoTime();

    cursor = testCache.query(query.setArgs(i));

    for(Entry<Integer,DataPOJO> entry : cursor){
        System.out.println("Entry fetched with key "+entry.getKey());
        totalTimeElapsed += System.nanoTime() - startTime;
    }
}

System.out.println("Total time taken to execute query is "+totalTimeElapsed+"ns");

所有查询的平均执行时间为 9 到 10 秒。

根据给定的数据，Ignite 能否在 100 到 200 毫秒内给出相同的结果？除了 Ignite 文档中提到的配置(我已经尝试过所有这些配置)之外，我可以在我的集群或缓存配置中进行配置以提高性能吗？

我不希望这是基于键的提取，而且我知道它比查询快得多。

Answer 1

我也遇到了同样的问题，比如查询无限运行。所以我将使用正确的索引在亚秒内运行查询所以请检查你的查询解释计划并检查查询执行计划中是否有任何完整的缓存扫描，如果你看到然后放置适当的个人或分组索引。如果您可以分享您的一个查询及其解释计划，那么我们可以为您提供有关它的模式想法。