Log the timestamps of each prefill/decode iteration #6094
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yuhsuan-t
requested review from
merrymercy,
Ying1123,
hnyls2002 and
xiezhq-hermann
as code owners
hebiao064
reviewed
May 9, 2025
| else: | ||
| f += f"#queue-req: {len(self.waiting_queue)}" | ||
|
|
||
| f += f", timestamp: {time.time()}" |
There was a problem hiding this comment.
How are you going to use this time? Just observe it from the log?
If that's the case, do you want to make it a human readable time format?
>>> import time
>>> print(time.time())
1746766086.594933
There was a problem hiding this comment.
Hello @hebiao064 , we actually have a script that parses the log, so the floating point makes sense to us. I can change it to human readable format if that's more useful. Thanks!
There was a problem hiding this comment.
Hello @hebiao064 I changed the logging to ISO 8601 format. Can you review again? Thanks!
There was a problem hiding this comment.
Hello @hebiao064 @Fridge003 , I have changed the code based on the review. Can you help review the PR again? Thank you :)
yuhsuan-t
force-pushed
the
yuhsuan-t/timestamp_logging
branch
from
7786e74 to
aa5d598
Compare
Logging the timestamps along with other stats helps with debugging performance issues
For better readability and parsability.
yuhsuan-t
force-pushed
the
yuhsuan-t/timestamp_logging
branch
from
aa5d598 to
4957840
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Logging the timestamps along with other stats helps with debugging performance issues
Modifications
Add timestamps to the prefill/decode logging functions.
Checklist