gpt4 book ai didi

C Tokenizer 程序中动态分配的内存 getNextToken 函数

转载 作者:行者123 更新时间:2023-11-30 14:23:50 26 4
gpt4 key购买 nike

下面是一个用于在进行标记化的程序中获取下一个标记的函数。它目前正在工作,但我仍然不确定它是否符合我的教授的要求。如果您查看顶部评论部分,他会说“返回 token 的空间应该动态分配。”

每当我听说我必须使用malloc,但我已经在没有使用malloc的情况下做到了。我的想法正确吗?

/*
* TKGetNextToken returns the next token from the token stream as a
* character string. Space for the returned token should be dynamically
* allocated. The caller is responsible for freeing the space once it is
* no longer needed.
*
* If the function succeeds, it returns a C string (delimited by '\0')
* containing the token. Else it returns 0.
*
* You need to fill in this function as part of your implementation.
*/
char *TKGetNextToken(TokenizerT *tk) {
char *sPtr, *tPtr, *delim, *temp = NULL, *ret = NULL;

sPtr = tk->sepr; //pointer to the separators

tPtr = tk->ts;
temp = tPtr;

while (tPtr[0] != '\0') //Scan tokenstream
{
delim = tk->sepr;
while (delim[0] != '\0') //scan separator stream
{
if (*tPtr == *delim) //Matched with a separator
{
if (tPtr == temp) //Check if beginning of the tokenstream
{
//then skip over this character.
temp++;
break; //Break loop because it may skip checking a char with a prev delim.
}
else
{
//Cut off current position with null character and pass over it.
*tPtr = '\0';
tPtr++;

if((ret = malloc(sizeof temp) + 1 * sizeof(char)) != NULL) //add 1 for null character '\0'
strcpy(ret, temp);

tk->ts = tPtr; //In position for next token.

return ret;
}
}
else
{
delim++; //Go to next separator.
}

}

tPtr++; //Go to next character.
count++;
}

if((ret = malloc(sizeof temp) + 1 * sizeof(char)) != NULL) //add 1 for null character '\0'
strcpy(ret, temp);

tk->ts = tPtr;
return ret;
}

编辑:所以我想我修复了它,但很好奇,按照我的方式使用 malloc,这不是只为指针地址而不是整个 token 分配空间吗?

最佳答案

这是一种可能的实现,但不是您的教授所要求的。您的教授希望您分配内存并将 token 复制到其中,然后返回指向该分配内存的指针。

关于C Tokenizer 程序中动态分配的内存 getNextToken 函数,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/12488746/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com