lua - 从 Redis 集中过滤/删除项目

Question

我将公交车站到达时间和路线 ID 缓存在一个 Redis 集合中，每个公交车站一组。我正在寻找一种简单的方法来更新这些集合，删除过去的到达时间，同时保留任何其他 future 的到达时间。如何在写入时根据特定条件过滤集合？

有一段时间没有写入的集会过期，所以我只关心不断更新的集，本质上是在元素级别而不是集级别设置过期。

Answer 1

以下将过滤掉小于 KEYS[2] 的值(在我的例子中是 UNIX 时间戳，因为 Redis 中的 LUA 脚本无法访问日期/时间信息而这样传递)。第二个子句然后添加传递给脚本的任何其他值。

local members_expired = 0 -- number of members expired/removed
local additions_attempted = 0 -- number of SADD attempts
local members_added = 0 -- number of members successfully added
local key = KEYS[1] -- the key of the set to update
local current_time = KEYS[2] -- the current timestamp

-- iterate through existing members and "expire" (remove) any members
-- less than the current time
for index, value in next, redis.call('SMEMBERS', key) do
    -- interpret the first 10 characters of the member as a timestamp,
    -- allowing us to include additional data such as the route ID
    if string.sub(value, 1, 10) < current_time then
        redis.call('SREM', key, value);
        members_expired = members_expired + 1
    end
end

-- iterate through provided members and attempt to insert them into the
-- target set
for index, value in next, ARGV do
    additions_attempted = additions_attempted + 1
    members_added = members_added + redis.call('SADD', key, value)
end

-- number of duplicate members
local duplicates_ignored = additions_attempted - members_added

-- entire set will expire in 1 week unless it's updated in the meantime
redis.call('EXPIRE', key, 604800)

return {
    members_added,
    members_expired,
    duplicates_ignored
}

脚本采用以下参数:

• “ key ”
  • 设置要更新的 key
  • 当前 UNIX 时间戳(无法从 Redis 内部访问，因此需要从外部传递)
• 值(value)观
  • 格式为[timestamp]:[extra_data]的一个或多个值，例如1474904925:route_123

它返回一个包含以下值的数组:

1. 添加到集合中的元素数。
2. 从集合中删除过去具有时间戳的元素数。
3. 插入失败的次数，假定是集合中已有的重复项。

PHP 示例(使用 Predis ):

$predis = new Predis\Client();
$time = time();

// some time in the future to add to the set
$values = [
    ($time + 3600) . ':route_123',
    ($time + 7200) . ':route_123',
    ($time + 7200) . ':route_456',
    ($time + 7200) . ':route_456', // this is a duplicate
];

$filter_script = <<<LUA
local members_expired = 0 -- number of members expired/removed
local additions_attempted = 0 -- number of SADD attempts
local members_added = 0 -- number of members successfully added
local key = KEYS[1] -- the key of the set to update
local current_time = KEYS[2] -- the current timestamp

-- iterate through existing members and "expire" (remove) any members
-- less than the current time
for index, value in next, redis.call('SMEMBERS', key) do
    -- interpret the first 10 characters of the member as a timestamp,
    -- allowing us to include additional data such as the route ID
    if string.sub(value, 1, 10) < current_time then
        redis.call('SREM', key, value);
        members_expired = members_expired + 1
    end
end

-- iterate through provided members and attempt to insert them into the
-- target set
for index, value in next, ARGV do
    additions_attempted = additions_attempted + 1
    members_added = members_added + redis.call('SADD', key, value)
end

-- number of duplicate members
local duplicates_ignored = additions_attempted - members_added

-- entire set will expire in 1 week unless it's updated in the meantime
redis.call('EXPIRE', key, 604800)

return {
    members_added,
    members_expired,
    duplicates_ignored
}
LUA;

// We can run the script directly...
list($members_added, $members_expired, $duplicates_ignored) = $predis->eval(
    $filter_script,
    2,
    'somekey',
    $time,
    $values[0],
    $values[1],
    $values[2],
    $values[3]
);

echo "Members added: $members_added\n";
echo "Members expired: $members_expired\n";
echo "Duplicate members ignored: $duplicates_ignored\n";
echo "\n";

// or save it for faster execution if we're going to run repeatedly
$members_added_total = 0;
$members_expired_total = 0;
$duplicates_ignored_total = 0;
$filter_script_sha = $predis->script('LOAD', $filter_script);

foreach ($values as $value) {
    list($members_added, $members_expired, $duplicates_ignored) =
        $predis->evalsha($filter_script_sha, 2, 'somekey', $time, $value);

    echo "[$members_added, $members_expired, $duplicates_ignored]\n";

    $members_added_total += $members_added;
    $members_expired_total += $members_expired;
    $duplicates_ignored_total += $duplicates_ignored;
}

echo "Members added: $members_added_total\n";
echo "Members expired: $members_expired_total\n";
echo "Duplicate members ignored: $duplicates_ignored_total\n";

eval 和 evalsha 的参数是:

1. 分别为脚本或 SHA1 哈希
2. 始终为“2”(传递给 KEYS LUA 变量的参数数量 - 参见 docs)
3. KEYS[1] 读取/修改的key
4. KEYS[2] 当前 UNIX 时间戳
5. (及以上)VALUES 要添加到集合中的任何新值