gpt4 book ai didi

macos - 为什么host_statistics64()会返回不一致的结果?

转载 作者:IT王子 更新时间:2023-10-28 23:30:25 48 4
gpt4 key购买 nike

为什么OS X 10.6.8中的host_statistics64()(我不知道其他版本是否有此问题)会返回不等于RAM总量的空闲、活动、非活动和有线内存计数?为什么它遗漏了不一致的页数?
以下输出表示10秒内未分类为空闲、活动、非活动或有线的页数(大约每秒采样一次)。

458
243
153
199
357
140
304
93
181
224

产生上述数字的代码是:
#include <stdio.h>
#include <mach/mach.h>
#include <mach/vm_statistics.h>
#include <sys/types.h>
#include <sys/sysctl.h>
#include <unistd.h>
#include <string.h>

int main(int argc, char** argv) {
struct vm_statistics64 stats;
mach_port_t host = mach_host_self();
natural_t count = HOST_VM_INFO64_COUNT;
natural_t missing = 0;
int debug = argc == 2 ? !strcmp(argv[1], "-v") : 0;
kern_return_t ret;
int mib[2];
long ram;
natural_t pages;
size_t length;
int i;

mib[0] = CTL_HW;
mib[1] = HW_MEMSIZE;
length = sizeof(long);
sysctl(mib, 2, &ram, &length, NULL, 0);
pages = ram / getpagesize();

for (i = 0; i < 10; i++) {
if ((ret = host_statistics64(host, HOST_VM_INFO64, (host_info64_t)&stats, &count)) != KERN_SUCCESS) {
printf("oops\n");
return 1;
}

/* updated for 10.9 */
missing = pages - (
stats.free_count +
stats.active_count +
stats.inactive_count +
stats.wire_count +
stats.compressor_page_count
);

if (debug) {
printf(
"%11d pages (# of pages)\n"
"%11d free_count (# of pages free) \n"
"%11d active_count (# of pages active) \n"
"%11d inactive_count (# of pages inactive) \n"
"%11d wire_count (# of pages wired down) \n"
"%11lld zero_fill_count (# of zero fill pages) \n"
"%11lld reactivations (# of pages reactivated) \n"
"%11lld pageins (# of pageins) \n"
"%11lld pageouts (# of pageouts) \n"
"%11lld faults (# of faults) \n"
"%11lld cow_faults (# of copy-on-writes) \n"
"%11lld lookups (object cache lookups) \n"
"%11lld hits (object cache hits) \n"
"%11lld purges (# of pages purged) \n"
"%11d purgeable_count (# of pages purgeable) \n"
"%11d speculative_count (# of pages speculative (also counted in free_count)) \n"
"%11lld decompressions (# of pages decompressed) \n"
"%11lld compressions (# of pages compressed) \n"
"%11lld swapins (# of pages swapped in (via compression segments)) \n"
"%11lld swapouts (# of pages swapped out (via compression segments)) \n"
"%11d compressor_page_count (# of pages used by the compressed pager to hold all the compressed data) \n"
"%11d throttled_count (# of pages throttled) \n"
"%11d external_page_count (# of pages that are file-backed (non-swap)) \n"
"%11d internal_page_count (# of pages that are anonymous) \n"
"%11lld total_uncompressed_pages_in_compressor (# of pages (uncompressed) held within the compressor.) \n",
pages, stats.free_count, stats.active_count, stats.inactive_count,
stats.wire_count, stats.zero_fill_count, stats.reactivations,
stats.pageins, stats.pageouts, stats.faults, stats.cow_faults,
stats.lookups, stats.hits, stats.purges, stats.purgeable_count,
stats.speculative_count, stats.decompressions, stats.compressions,
stats.swapins, stats.swapouts, stats.compressor_page_count,
stats.throttled_count, stats.external_page_count,
stats.internal_page_count, stats.total_uncompressed_pages_in_compressor
);
}

printf("%i\n", missing);
sleep(1);
}

return 0;
}

最佳答案

DR:
host_statistics64()从不同来源获取信息,这可能会花费时间,并可能产生不一致的结果。
host_statistics64()通过名为vm_page_foo_count的变量获取一些信息。但并非所有这些变量都被考虑在内,例如,vm_page_stolen_count不是。
众所周知,/usr/bin/top会将失窃的页面添加到有线页面的数量中。这是一个指标,说明在计算页面时应考虑这些页面。
笔记
我正在开发一款带有达尔文内核版本16.5.0 xnu-3789.51.2~3/release_x86_64 x86_64的MacOS 10.12,但所有行为都是完全可复制的。
我要链接很多我在机器上使用的XNU版本的源代码。可以在这里找到:xnu-3789.51.2
您编写的程序基本上与/usr/bin/vm_stat相同,后者只是host_statistics64()(andhost_statistics())的包装器。相关源代码可以在以下位置找到:system_cmds-496/vm_stat.tproj/vm_stat.c
host_statistics64()如何适应xnu,它是如何工作的?
正如Wistle所知道的,OS X内核被称为XNU(XNU不是UNIX),“是一个混合内核,它结合了卡内基梅隆大学开发的Mach内核和FreeBSD和C++API编写IOKit的驱动程序的组件”(https://github.com/opensource-apple/xnu/blob/10.12/README.md)。
虚拟内存管理(VM)是mach的一部分,因此host_statistics64()位于此处。让我们更详细地了解一下xnu-3789.51.2/osfmk/kern/host.c中包含的ITS实现。
函数签名是

kern_return_t
host_statistics64(host_t host, host_flavor_t flavor, host_info64_t info, mach_msg_type_number_t * count);

第一个相关行是
[...]
processor_t processor;
vm_statistics64_t stat;
vm_statistics64_data_t host_vm_stat;
mach_msg_type_number_t original_count;
unsigned int local_q_internal_count;
unsigned int local_q_external_count;
[...]
processor = processor_list;
stat = &PROCESSOR_DATA(processor, vm_stat);
host_vm_stat = *stat;

if (processor_count > 1) {
simple_lock(&processor_list_lock);

while ((processor = processor->processor_list) != NULL) {
stat = &PROCESSOR_DATA(processor, vm_stat);

host_vm_stat.zero_fill_count += stat->zero_fill_count;
host_vm_stat.reactivations += stat->reactivations;
host_vm_stat.pageins += stat->pageins;
host_vm_stat.pageouts += stat->pageouts;
host_vm_stat.faults += stat->faults;
host_vm_stat.cow_faults += stat->cow_faults;
host_vm_stat.lookups += stat->lookups;
host_vm_stat.hits += stat->hits;
host_vm_stat.compressions += stat->compressions;
host_vm_stat.decompressions += stat->decompressions;
host_vm_stat.swapins += stat->swapins;
host_vm_stat.swapouts += stat->swapouts;
}

simple_unlock(&processor_list_lock);
}
[...]

我们得到的是 host_vm_stat类型。这只是一个 vm_statistics64_data_t如您在 xnu-3789.51.2/osfmk/mach/vm_statistics.h中看到的。我们从 xnu-3789.51.2/osfmk/kern/processor_data.h中定义的makro typedef struct vm_statistics64中获取处理器信息。我们只需将相关的数字相加,就可以在循环遍历所有处理器的同时填充 PROCESSOR_DATA()
如您所见,我们发现了一些众所周知的统计数据,如 host_vm_statzero_fill_count但并非所有的统计数据都包含在 compressions中。
接下来的相关行是:
stat = (vm_statistics64_t)info;

stat->free_count = vm_page_free_count + vm_page_speculative_count;
stat->active_count = vm_page_active_count;
[...]
stat->inactive_count = vm_page_inactive_count;
stat->wire_count = vm_page_wire_count + vm_page_throttled_count + vm_lopage_free_count;
stat->zero_fill_count = host_vm_stat.zero_fill_count;
stat->reactivations = host_vm_stat.reactivations;
stat->pageins = host_vm_stat.pageins;
stat->pageouts = host_vm_stat.pageouts;
stat->faults = host_vm_stat.faults;
stat->cow_faults = host_vm_stat.cow_faults;
stat->lookups = host_vm_stat.lookups;
stat->hits = host_vm_stat.hits;

stat->purgeable_count = vm_page_purgeable_count;
stat->purges = vm_page_purged_count;

stat->speculative_count = vm_page_speculative_count;

我们重用并使其成为我们的输出结构。然后,我们将两个称为 host_statistics64()statfree_count相加。我们以同样的方式收集其他剩余的数据(使用名为 unsigned long的变量)或从上面填写的 vm_page_free_count中获取统计信息。
1。结论我们收集了不同来源的数据。要么来自处理器信息,要么来自称为 vm_page_speculative_count的变量。这需要时间,并且可能会以一些不符合实际的事情结束——事实上,虚拟机是一个非常快速和连续的过程。
让我们仔细看看已经提到的变量。它们在 xnu-3789.51.2/osfmk/vm/vm_page.h中定义如下:
extern
unsigned int vm_page_free_count; /* How many pages are free? (sum of all colors) */
extern
unsigned int vm_page_active_count; /* How many pages are active? */
extern
unsigned int vm_page_inactive_count; /* How many pages are inactive? */
#if CONFIG_SECLUDED_MEMORY
extern
unsigned int vm_page_secluded_count; /* How many pages are secluded? */
extern
unsigned int vm_page_secluded_count_free;
extern
unsigned int vm_page_secluded_count_inuse;
#endif /* CONFIG_SECLUDED_MEMORY */
extern
unsigned int vm_page_cleaned_count; /* How many pages are in the clean queue? */
extern
unsigned int vm_page_throttled_count;/* How many inactives are throttled */
extern
unsigned int vm_page_speculative_count; /* How many speculative pages are unclaimed? */
extern unsigned int vm_page_pageable_internal_count;
extern unsigned int vm_page_pageable_external_count;
extern
unsigned int vm_page_xpmapped_external_count; /* How many pages are mapped executable? */
extern
unsigned int vm_page_external_count; /* How many pages are file-backed? */
extern
unsigned int vm_page_internal_count; /* How many pages are anonymous? */
extern
unsigned int vm_page_wire_count; /* How many pages are wired? */
extern
unsigned int vm_page_wire_count_initial; /* How many pages wired at startup */
extern
unsigned int vm_page_free_target; /* How many do we want free? */
extern
unsigned int vm_page_free_min; /* When to wakeup pageout */
extern
unsigned int vm_page_throttle_limit; /* When to throttle new page creation */
extern
uint32_t vm_page_creation_throttle; /* When to throttle new page creation */
extern
unsigned int vm_page_inactive_target;/* How many do we want inactive? */
#if CONFIG_SECLUDED_MEMORY
extern
unsigned int vm_page_secluded_target;/* How many do we want secluded? */
#endif /* CONFIG_SECLUDED_MEMORY */
extern
unsigned int vm_page_anonymous_min; /* When it's ok to pre-clean */
extern
unsigned int vm_page_inactive_min; /* When to wakeup pageout */
extern
unsigned int vm_page_free_reserved; /* How many pages reserved to do pageout */
extern
unsigned int vm_page_throttle_count; /* Count of page allocations throttled */
extern
unsigned int vm_page_gobble_count;
extern
unsigned int vm_page_stolen_count; /* Count of stolen pages not acccounted in zones */
[...]
extern
unsigned int vm_page_purgeable_count;/* How many pages are purgeable now ? */
extern
unsigned int vm_page_purgeable_wired_count;/* How many purgeable pages are wired now ? */
extern
uint64_t vm_page_purged_count; /* How many pages got purged so far ? */

这是很多关于我们只能使用 vm_page_foo_count访问非常有限数量的统计数据。大多数统计信息都在 xnu-3789.51.2/osfmk/vm/vm_resident.c中更新。例如,此函数将页面释放到可用页面列表中:
/*
* vm_page_release:
*
* Return a page to the free list.
*/

void
vm_page_release(
vm_page_t mem,
boolean_t page_queues_locked)
{
[...]
vm_page_free_count++;
[...]
}

非常有趣的是 host_vm_stat。什么是失窃的页面?似乎有一些机制可以从一些列表中取出一个页面,即使它通常不会被调出。其中一种机制是推测性页面列表中页面的年龄。告诉我们
* VM_PAGE_MAX_SPECULATIVE_AGE_Q * VM_PAGE_SPECULATIVE_Q_AGE_MS
* defines the amount of time a speculative page is normally
* allowed to live in the 'protected' state (i.e. not available
* to be stolen if vm_pageout_scan is running and looking for
* pages)... however, if the total number of speculative pages
* in the protected state exceeds our limit (defined in vm_pageout.c)
* and there are none available in VM_PAGE_SPECULATIVE_AGED_Q, then
* vm_pageout_scan is allowed to steal pages from the protected
* bucket even if they are underage.
*
* vm_pageout_scan is also allowed to pull pages from a protected
* bin if the bin has reached the "age of consent" we've set

实际上, vm_page_foo_count是递增的。您可以在 xnu-3789.51.2/osfmk/vm/vm_page.h中找到相应的源代码。
我认为在计算vm stats a vm_page_foo_count时不考虑失窃的页面。
证明我是对的证据
证明这一点的最好方法是用手工定制的版本来编译xnu。我没有机会这么做,但很快就会尝试。
幸运的是,我们不是唯一对正确的VM统计感兴趣的人。因此,我们应该看看Well know(不包含在xnu中)的实现,它在这里完全可用: xnu-3789.51.2/osfmk/vm/vm_pageout.c(我刚刚选择了 top-108)。
让我们来看一下 macOS 10.12.4 release中的以下内容:
static int
libtop_tsamp_update_vm_stats(libtop_tsamp_t* tsamp) {
kern_return_t kr;
tsamp->p_vm_stat = tsamp->vm_stat;

mach_msg_type_number_t count = sizeof(tsamp->vm_stat) / sizeof(natural_t);
kr = host_statistics64(libtop_port, HOST_VM_INFO64, (host_info64_t)&tsamp->vm_stat, &count);
if (kr != KERN_SUCCESS) {
return kr;
}

if (tsamp->pages_stolen > 0) {
tsamp->vm_stat.wire_count += tsamp->pages_stolen;
}

[...]

return kr;
}

host_statistics64()属于 extern unsigned int vm_page_stolen_count; /* Count of stolen pages not acccounted in zones */类型,它是在 top-108/libtop.c中定义的结构。它包含其他内容 void vm_pageout_scan(void)vm_page_stolen_count
如你所见, host_statistics64()被我们所知的 host_statistics64()填充。然后它检查是否 /usr/bin/top并将其添加到 tsamplibtop_tsamp_t字段中。
2。结论如果我们只使用 vm_statistics64_data_t vm_stat作为 uint64_t pages_stolen或您的示例代码,我们将无法获得这些被盗页的数量!
为什么要按原样实施 static int libtop_tsamp_update_vm_stats(libtop_tsamp_t* tsamp)
老实说,我不知道。分页是一个复杂的过程,因此实时观察是一项具有挑战性的任务。我们必须注意到它的实现中似乎没有bug。我认为,如果我们能够访问 tsamp->vm_stat的话,我们甚至无法获得100%准确的页面数量。如果失窃的页数不是很大的话, host_statistics64()的实现不会计算失窃的页数。
另一个有趣的事情是在函数 tsamp->pages_stolen > 0上面的注释,它是 wire_counttop-108/libtop.h是一个苹果软件的bug报告站点,通常按照注释中给出的格式对bug进行分类。我找不到相关的bug;可能是关于缺页。
我希望这些信息能对你有所帮助。如果我能在我的机器上编译XNU的最新(和定制)版本,我会告诉你。也许这会带来有趣的见解。

关于macos - 为什么host_statistics64()会返回不一致的结果?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/14789672/

48 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com