c++ - 字符串中的重音，如何将重音字符存储在单个字符中

Question

我必须编写一个小程序来从输入的字符串中删除重音符号。我还必须创建一个函数，将每个带有重音的字符替换为相应的不带重音的字符，并且我有一个循环为我的 main 中的每个字符调用该函数:

char func (char c)
{
    string acc = "èé";
    string norm = "ee";
    char ret = c;

    for(int i = 0; i < acc.size(); i++)
    {
        if(c == acc[i])
            ret = acc[i];
    }
    return ret;
}

问题是，如果我在 main 中提供字符串“é”作为输入，该字符串将被视为大小为 2 的字符串(参见下面的示例)，并且上面的函数被调用两次而不是一次。此外，作为函数输入给出的 char 不是正确的。我想我的函数内部有同样大小的问题。这个重音不应该被看作是一个单独的字符吗？ (我使用的是 UTF-8)

string s = "e";
cout << "size:" << s.size() << endl;
s = "è";
cout << "size:" << s.size() << endl;

OUTPUT
size:1
size:2

我已经使用 wchar_t 和 wstring 类型解决了这个问题，但是我需要将这个函数插入到一个更复杂的程序中，并且我可能想避免更改所有代码来处理 wstring。

我需要更改文件编码吗？实际的是:

text/x-c; charset=utf-8

是否可以使用普通字符串和字符编写这样的函数？

Answer 1

您不应尝试使用简单的循环自行执行此操作，尤其是在代码对安全敏感的情况下。 There is often more than one way to represent the same character in Unicode ，因此您可能只有一个代码点，也可能有两个代码点。例如:

const wchar_t text1[] = { L'e', 0x0301, 0 };
const wchar_t text2[] = { 0x0e9, 0 };

这两个字符串在打印时看起来是等价的(它们都显示 é)，但它们显然不同，简单的 == 检查将失败。您应该在搜索之前规范化字符串，或使用自动为您执行此操作的现有函数。

Windows 提供了NormalizeString 和FindStringOrdinal，ICU 为此提供了unorm_compare 或usearch_first。

const wchar_t text1[] = { L'e', 0x0301, 0 };
const wchar_t text2[] = { 0x0e9, 0 };

// Using Windows APIs, try to normalize the string first
int size = NormalizeString(NormalizationKC, text1, -1, nullptr, 0);
if (size == 0)
  throw std::exception("Can't normalize");

auto text3 = std::make_unique<wchar_t[]>(size);
NormalizeString(NormalizationKC, text1, -1, text3.get(), size);

// Print out the three strings - they all look the same
std::wcout << text1 << std::endl;
std::wcout << text2 << std::endl;
std::wcout << text3.get() << std::endl;

// Verify if they are (or are not) equal
if (CompareStringOrdinal(text1, -1, text2, -1, false) == 2)
  std::wcout << L"Original strings are equivalent\r\n";
else
  std::wcout << L"Original strings are not equivalent\r\n";

if (CompareStringOrdinal(text3.get(), -1, text2, -1, false) == 2)
  std::wcout << L"Normalized strings are equivalent\r\n";
else
  std::wcout << L"Normalized strings are not equivalent\r\n";

// Verify if the string text2 can be found
if (FindStringOrdinal(FIND_FROMSTART, text1, -1, text2, -1, TRUE) != -1)
  std::wcout << L"Original string contains the searched-for string\r\n";
else
  std::wcout << L"Original string does not contain the searched-for string\r\n";

if (FindStringOrdinal(FIND_FROMSTART, text3.get(), -1, text2, -1, TRUE) != -1)
  std::wcout << L"Normalized string contains the searched-for string\r\n";
else
  std::wcout << L"Normalized string does not contain the searched-for string\r\n";

// Using ICU APIs, try to compare the normalized strings in one go
// (You can also manually normalize, like Windows, if you want to keep the
// normalized form around)
UErrorCode error{ U_ZERO_ERROR };
auto result = unorm_compare(reinterpret_cast<const UChar*>(text1), -1, 
  reinterpret_cast<const UChar*>(text2), -1, 0, &error);
if (!U_SUCCESS(error))
  throw std::exception("Can't normalize");

if (result == 0)
  std::wcout << L"[ICU] Normalized strings are equivalent\r\n";
else
  std::wcout << L"[ICU] Normalized strings are NOT equivalent\r\n";

// Try searching; ICU handles the equivalency of (non-)normalized
// characters automatically.
auto search = usearch_open(reinterpret_cast<const UChar*>(text2), -1, 
  reinterpret_cast<const UChar*>(text1), -1, "", nullptr, &error);
if (!U_SUCCESS(error))
  throw std::exception("Can't open search");

auto index = usearch_first(search, &error);
if (!U_SUCCESS(error))
  throw std::exception("Can't search");

if (index != USEARCH_DONE)
  std::wcout << L"[ICU] Original string contains the searched-for string\r\n";
else
  std::wcout << L"[ICU] Original string does not contain the searched-for string\r\n";

usearch_close(search);

这会产生以下输出:

é
é
é
Original strings are not equivalent
Normalized strings are equivalent
Original string does not contain the searched-for string
Normalized string contains the searched-for string
[ICU] Normalized strings are equivalent
[ICU] Original string contains the searched-for string