-
Notifications
You must be signed in to change notification settings - Fork 492
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add zfree_with_size #453
base: unstable
Are you sure you want to change the base?
Add zfree_with_size #453
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## unstable #453 +/- ##
============================================
+ Coverage 70.07% 70.23% +0.15%
============================================
Files 110 110
Lines 59989 59994 +5
============================================
+ Hits 42037 42134 +97
+ Misses 17952 17860 -92
|
I think the zmalloc_size is really expensive to update the |
@lipzhu the update of used_memory has 2 cost aspects, the zmalloc_size and also the fact it's an atomic operation.
|
@lipzhu good point about |
src/sds.c
Outdated
@@ -39,6 +39,7 @@ | |||
#include "sds.h" | |||
#include "sdsalloc.h" | |||
#include "util.h" | |||
#include "zmalloc.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't really want zmalloc in sds, since it uses sdsalloc.
EDIT: I see now we call zmalloc in this file anyways, and probably should have been imported before, but we would still like not have a dependency on zmalloc in this file.
…to align the logic with sdsnewlen function. (valkey-io#476) This patch try to correct the actual allocated size from allocator when call sdsRedize to align the logic with sdsnewlen function. Maybe the valkey-io#453 optimization should depend on this. Signed-off-by: Lipeng Zhu <lipeng.zhu@intel.com>
…to align the logic with sdsnewlen function. (valkey-io#476) This patch try to correct the actual allocated size from allocator when call sdsRedize to align the logic with sdsnewlen function. Maybe the valkey-io#453 optimization should depend on this. Signed-off-by: Lipeng Zhu <lipeng.zhu@intel.com> Signed-off-by: Samuel Adetunji <adetunjithomas1@outlook.com>
zfree updates memory statistics. It gets the size of the buffer from jemalloc by calling zmalloc_size. This operation is costly. We can avoid it if we know the buffer size. For example, we can calculate size of sds from the data we have in its header. This commit introduces zfree_with_size function that accepts both pointer to a buffer, and its size. zfree is refactored to call zfree_with_size. sdsfree uses the new interface for all but SDS_TYPE_5. Signed-off-by: Vadym Khoptynets <vadymkh@amazon.com>
sdsalloc returns sds's length for SDS_TYPE_5. It's not correct for SDS_TYPE_5. This patch makes sdsAllocSize call zmalloc_size for SDS_TYPE_5. sdsalloc is a lower level function that continues to return length for SDS_TYPE_5. Signed-off-by: Vadym Khoptynets <vadymkh@amazon.com>
Instead of zmalloc_size use s_malloc_size. Signed-off-by: Vadym Khoptynets <vadymkh@amazon.com>
Description
zfree updates memory statistics. It gets the size of the buffer from jemalloc by calling zmalloc_size. This operation is costly. We can avoid it if we know the buffer size. For example, we can calculate size of sds from the data we have in its header.
This commit introduces zfree_with_size function that accepts both pointer to a buffer, and its size. zfree is refactored to call zfree_with_size.
sdsfree uses the new interface for all but SDS_TYPE_5.
Benchmark
Dataset is 3 million strings. Each benchmark run uses its own value size (8192, 512, and 120). The benchmark is 100% write load for 5 minutes.