I want to tokenize a c program into stream of tokens like identifiers, operators, separators, keywords, comments etc. I do not know which tokenizer or compiler used for this purpose. Please guide me which tokenizer is used for this purpose and how it works? The c program I want to tokenize is given below.

void sumProd1A(int n) {
    double sum = 0.0; //C1
    double prod = 1.0;
    int i;
    for (i = 1; i <= n; i++)
        sum = sum + i;
        prod = prod * i;
        foo2(sum, prod);
void main()
    printf("\nsumProd1A_Cordy: %lf ");

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.