引言 - 导航栏目
有些朋友可能对 redis 充满着数不尽的求知欲, 也许是 redis 属于工作, 交流(面试)的大头戏,
不得不 ... 而自己当下对于 redis 只是停留在会用层面, 细节层面几乎没有涉猎. 为了更快的融于大
家, 这里尝试抛砖引玉. 先带大家手写个 redis 中最简单的数据结构, adlist 双向链表. 让我们一
起对 redis 有个初步的认知. 本文会从下面几个标题展开解读(吐槽), 欢迎交流和指正.
1. redis adlist 解析
2. redis config.h 分析
3. redis setproctitle.c 分析
4. redis atomicvar.h 分析
5. redis zmalloc 分析
6. redis Makefile 解析
redis 大头是 C 写的, 而 C 啥也不缺, 就缺手写, OK 开始废话手写之旅吧 🙂
前言 - redis adlist
全篇示例代码都有手写过, 不过为了素材正规, 这里直接原封不动的引用
github.com/antirez/redis 中相关代码.
1. redis adlist 解析
/* adlist.h - A generic doubly linked list implementation
*
* Copyright (c) 2006-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/ #ifndef __ADLIST_H__
#define __ADLIST_H__ /* Node, List, and Iterator are the only data structures used currently. */ typedef struct listNode {
struct listNode *prev;
struct listNode *next;
void *value;
} listNode; typedef struct listIter {
listNode *next;
int direction;
} listIter; typedef struct list {
listNode *head;
listNode *tail;
void *(*dup)(void *ptr);
void (*free)(void *ptr);
int (*match)(void *ptr, void *key);
unsigned long len;
} list; /* Functions implemented as macros */
#define listLength(l) ((l)->len)
#define listFirst(l) ((l)->head)
#define listLast(l) ((l)->tail)
#define listPrevNode(n) ((n)->prev)
#define listNextNode(n) ((n)->next)
#define listNodeValue(n) ((n)->value) #define listSetDupMethod(l,m) ((l)->dup = (m))
#define listSetFreeMethod(l,m) ((l)->free = (m))
#define listSetMatchMethod(l,m) ((l)->match = (m)) #define listGetDupMethod(l) ((l)->dup)
#define listGetFreeMethod(l) ((l)->free)
#define listGetMatchMethod(l) ((l)->match) /* Prototypes */
list *listCreate(void);
void listRelease(list *list);
void listEmpty(list *list);
list *listAddNodeHead(list *list, void *value);
list *listAddNodeTail(list *list, void *value);
list *listInsertNode(list *list, listNode *old_node, void *value, int after);
void listDelNode(list *list, listNode *node);
listIter *listGetIterator(list *list, int direction);
listNode *listNext(listIter *iter);
void listReleaseIterator(listIter *iter);
list *listDup(list *orig);
listNode *listSearchKey(list *list, void *key);
listNode *listIndex(list *list, long index);
void listRewind(list *list, listIter *li);
void listRewindTail(list *list, listIter *li);
void listRotate(list *list);
void listJoin(list *l, list *o); /* Directions for iterators */
#define AL_START_HEAD 0
#define AL_START_TAIL 1 #endif /* __ADLIST_H__ */
首先手写的是 adlist.h 双向链表的头文件, 对于这个头文件有几点要聊一聊的.
1.1' redis 中头文件格式目前没有统一
#ifndef __ADLIST_H__ #endif
#ifndef __REDIS_HELP_H #endif
#ifndef __ZMALLOC_H #endif
可能也是, redis 这个项目维护和开发都十年多了. 代码风格在变(千奇百怪)也是正常.
这里推荐第三种写法 -> __{不带后缀文件名}_H
1.2' adlist.h 中函数命名随意
void listReleaseIterator(listIter *iter);
list *listDup(list *orig);
listNode *listSearchKey(list *list, void *key);
listNode *listIndex(list *list, long index);
void listRewind(list *list, listIter *li);
void listRewindTail(list *list, listIter *li);
void listRotate(list *list);
void listJoin(list *l, list *o);
命名随意不是个好习惯, 推荐参数名强区分. 例如下面这样固定格式
extern void listReleaseIterator(listIter * iter);
extern list * listDup(list * l);
extern listNode * listSearchKey(list * l, void * key);
extern listNode * listIndex(list * l, long index);
extern void listRewind(list * l, listIter * iter);
extern void listRewindTail(list * l, listIter * iter);
extern void listRotate(list * l);
extern void listJoin(list * l, list * o);
写完 adlist.h 接口定义部分, 相信有些人对待实现的 adlist.c 也有了大致轮廓了吧 🙂
/* adlist.c - A generic doubly linked list implementation
*
* Copyright (c) 2006-2010, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/ #include <stdlib.h>
#include "adlist.h"
#include "zmalloc.h" /* Create a new list. The created list can be freed with
* AlFreeList(), but private value of every node need to be freed
* by the user before to call AlFreeList().
*
* On error, NULL is returned. Otherwise the pointer to the new list. */
list *listCreate(void)
{
struct list *list; if ((list = zmalloc(sizeof(*list))) == NULL)
return NULL;
list->head = list->tail = NULL;
list->len = ;
list->dup = NULL;
list->free = NULL;
list->match = NULL;
return list;
} /* Remove all the elements from the list without destroying the list itself. */
void listEmpty(list *list)
{
unsigned long len;
listNode *current, *next; current = list->head;
len = list->len;
while(len--) {
next = current->next;
if (list->free) list->free(current->value);
zfree(current);
current = next;
}
list->head = list->tail = NULL;
list->len = ;
} /* Free the whole list.
*
* This function can't fail. */
void listRelease(list *list)
{
listEmpty(list);
zfree(list);
} /* Add a new node to the list, to head, containing the specified 'value'
* pointer as value.
*
* On error, NULL is returned and no operation is performed (i.e. the
* list remains unaltered).
* On success the 'list' pointer you pass to the function is returned. */
list *listAddNodeHead(list *list, void *value)
{
listNode *node; if ((node = zmalloc(sizeof(*node))) == NULL)
return NULL;
node->value = value;
if (list->len == ) {
list->head = list->tail = node;
node->prev = node->next = NULL;
} else {
node->prev = NULL;
node->next = list->head;
list->head->prev = node;
list->head = node;
}
list->len++;
return list;
} /* Add a new node to the list, to tail, containing the specified 'value'
* pointer as value.
*
* On error, NULL is returned and no operation is performed (i.e. the
* list remains unaltered).
* On success the 'list' pointer you pass to the function is returned. */
list *listAddNodeTail(list *list, void *value)
{
listNode *node; if ((node = zmalloc(sizeof(*node))) == NULL)
return NULL;
node->value = value;
if (list->len == ) {
list->head = list->tail = node;
node->prev = node->next = NULL;
} else {
node->prev = list->tail;
node->next = NULL;
list->tail->next = node;
list->tail = node;
}
list->len++;
return list;
} list *listInsertNode(list *list, listNode *old_node, void *value, int after) {
listNode *node; if ((node = zmalloc(sizeof(*node))) == NULL)
return NULL;
node->value = value;
if (after) {
node->prev = old_node;
node->next = old_node->next;
if (list->tail == old_node) {
list->tail = node;
}
} else {
node->next = old_node;
node->prev = old_node->prev;
if (list->head == old_node) {
list->head = node;
}
}
if (node->prev != NULL) {
node->prev->next = node;
}
if (node->next != NULL) {
node->next->prev = node;
}
list->len++;
return list;
} /* Remove the specified node from the specified list.
* It's up to the caller to free the private value of the node.
*
* This function can't fail. */
void listDelNode(list *list, listNode *node)
{
if (node->prev)
node->prev->next = node->next;
else
list->head = node->next;
if (node->next)
node->next->prev = node->prev;
else
list->tail = node->prev;
if (list->free) list->free(node->value);
zfree(node);
list->len--;
} /* Returns a list iterator 'iter'. After the initialization every
* call to listNext() will return the next element of the list.
*
* This function can't fail. */
listIter *listGetIterator(list *list, int direction)
{
listIter *iter; if ((iter = zmalloc(sizeof(*iter))) == NULL) return NULL;
if (direction == AL_START_HEAD)
iter->next = list->head;
else
iter->next = list->tail;
iter->direction = direction;
return iter;
} /* Release the iterator memory */
void listReleaseIterator(listIter *iter) {
zfree(iter);
} /* Create an iterator in the list private iterator structure */
void listRewind(list *list, listIter *li) {
li->next = list->head;
li->direction = AL_START_HEAD;
} void listRewindTail(list *list, listIter *li) {
li->next = list->tail;
li->direction = AL_START_TAIL;
} /* Return the next element of an iterator.
* It's valid to remove the currently returned element using
* listDelNode(), but not to remove other elements.
*
* The function returns a pointer to the next element of the list,
* or NULL if there are no more elements, so the classical usage patter
* is:
*
* iter = listGetIterator(list,<direction>);
* while ((node = listNext(iter)) != NULL) {
* doSomethingWith(listNodeValue(node));
* }
*
* */
listNode *listNext(listIter *iter)
{
listNode *current = iter->next; if (current != NULL) {
if (iter->direction == AL_START_HEAD)
iter->next = current->next;
else
iter->next = current->prev;
}
return current;
} /* Duplicate the whole list. On out of memory NULL is returned.
* On success a copy of the original list is returned.
*
* The 'Dup' method set with listSetDupMethod() function is used
* to copy the node value. Otherwise the same pointer value of
* the original node is used as value of the copied node.
*
* The original list both on success or error is never modified. */
list *listDup(list *orig)
{
list *copy;
listIter iter;
listNode *node; if ((copy = listCreate()) == NULL)
return NULL;
copy->dup = orig->dup;
copy->free = orig->free;
copy->match = orig->match;
listRewind(orig, &iter);
while((node = listNext(&iter)) != NULL) {
void *value; if (copy->dup) {
value = copy->dup(node->value);
if (value == NULL) {
listRelease(copy);
return NULL;
}
} else
value = node->value;
if (listAddNodeTail(copy, value) == NULL) {
listRelease(copy);
return NULL;
}
}
return copy;
} /* Search the list for a node matching a given key.
* The match is performed using the 'match' method
* set with listSetMatchMethod(). If no 'match' method
* is set, the 'value' pointer of every node is directly
* compared with the 'key' pointer.
*
* On success the first matching node pointer is returned
* (search starts from head). If no matching node exists
* NULL is returned. */
listNode *listSearchKey(list *list, void *key)
{
listIter iter;
listNode *node; listRewind(list, &iter);
while((node = listNext(&iter)) != NULL) {
if (list->match) {
if (list->match(node->value, key)) {
return node;
}
} else {
if (key == node->value) {
return node;
}
}
}
return NULL;
} /* Return the element at the specified zero-based index
* where 0 is the head, 1 is the element next to head
* and so on. Negative integers are used in order to count
* from the tail, -1 is the last element, -2 the penultimate
* and so on. If the index is out of range NULL is returned. */
listNode *listIndex(list *list, long index) {
listNode *n; if (index < ) {
index = (-index)-;
n = list->tail;
while(index-- && n) n = n->prev;
} else {
n = list->head;
while(index-- && n) n = n->next;
}
return n;
} /* Rotate the list removing the tail node and inserting it to the head. */
void listRotate(list *list) {
listNode *tail = list->tail; if (listLength(list) <= ) return; /* Detach current tail */
list->tail = tail->prev;
list->tail->next = NULL;
/* Move it as head */
list->head->prev = tail;
tail->prev = NULL;
tail->next = list->head;
list->head = tail;
} /* Add all the elements of the list 'o' at the end of the
* list 'l'. The list 'other' remains empty but otherwise valid. */
void listJoin(list *l, list *o) {
if (o->head)
o->head->prev = l->tail; if (l->tail)
l->tail->next = o->head;
else
l->head = o->head; if (o->tail) l->tail = o->tail;
l->len += o->len; /* Setup other as an empty list. */
o->head = o->tail = NULL;
o->len = ;
}
是的, 就是这样, 就是这样简单.
我们稍微多讲点, 其实对于 listCreate 可以写的更加简约, 不是吗?
struct list * listCreate(void) {
return zcalloc(sizeof(struct list));
}
好了, 那我们继续交流(吐槽)吧.
1.3' 代码括号 { } 位置随意
这不是个好习惯, 毕竟谁也不喜欢两面派. 大项目还是得需要在大方向上统一风格和约束.
1.4' struct listIter::direction 不一定是个很好的设计
direction 通过与 AL_START_HEAD or AL_START_TAIL 宏进行运行时比对, 来区分遍历的方向. 觉得
有点浪费. 内心更倾向于干掉运行时比对, 从一开始用户就应该知道该怎么遍历更好, 毕竟这是所有数据结构
的标杆.
❤ 恭喜大家, 到这我们关于 redis adlist 最基础最简单的数据结构已经手写分析完毕, 后面可以不用看了.
谢谢大家捧场 ~
正文 - adlist 周边
简单愉快的背后总会有些更深的不可捉摸. 离开了奶头乐, 我们将从 adlist.c 中一行代码, 正式开启
我们此次探险之旅.
#include "zmalloc.h"
2. redis config.h 分析
同样在 zmallo.c 中我们发现了如下两行代码, 这就是我们要说的一个主体之一 config.h
#include "config.h"
#include "atomicvar.h"
config.h 主要作用是用于确定程序的运行环境, 例如是什么操作系统, 是什么字节序, 要不要启用某些功能
/*
* Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/ #ifndef __CONFIG_H
#define __CONFIG_H #ifdef __APPLE__
#include <AvailabilityMacros.h>
#endif #ifdef __linux__
#include <linux/version.h>
#include <features.h>
#endif /* Define redis_fstat to fstat or fstat64() */
#if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#define redis_fstat fstat64
#define redis_stat stat64
#else
#define redis_fstat fstat
#define redis_stat stat
#endif /* Test for proc filesystem */
#ifdef __linux__
#define HAVE_PROC_STAT 1
#define HAVE_PROC_MAPS 1
#define HAVE_PROC_SMAPS 1
#define HAVE_PROC_SOMAXCONN 1
#endif /* Test for task_info() */
#if defined(__APPLE__)
#define HAVE_TASKINFO 1
#endif /* Test for backtrace() */
#if defined(__APPLE__) || (defined(__linux__) && defined(__GLIBC__)) || \
defined(__FreeBSD__) || (defined(__OpenBSD__) && defined(USE_BACKTRACE))\
|| defined(__DragonFly__)
#define HAVE_BACKTRACE 1
#endif /* MSG_NOSIGNAL. */
#ifdef __linux__
#define HAVE_MSG_NOSIGNAL 1
#endif /* Test for polling API */
#ifdef __linux__
#define HAVE_EPOLL 1
#endif #if (defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)) || defined(__FreeBSD__) || defined(__OpenBSD__) || defined (__NetBSD__)
#define HAVE_KQUEUE 1
#endif #ifdef __sun
#include <sys/feature_tests.h>
#ifdef _DTRACE_VERSION
#define HAVE_EVPORT 1
#endif
#endif /* Define redis_fsync to fdatasync() in Linux and fsync() for all the rest */
#ifdef __linux__
#define redis_fsync fdatasync
#else
#define redis_fsync fsync
#endif /* Define rdb_fsync_range to sync_file_range() on Linux, otherwise we use
* the plain fsync() call. */
#ifdef __linux__
#if defined(__GLIBC__) && defined(__GLIBC_PREREQ)
#if (LINUX_VERSION_CODE >= 0x020611 && __GLIBC_PREREQ(2, 6))
#define HAVE_SYNC_FILE_RANGE 1
#endif
#else
#if (LINUX_VERSION_CODE >= 0x020611)
#define HAVE_SYNC_FILE_RANGE 1
#endif
#endif
#endif #ifdef HAVE_SYNC_FILE_RANGE
#define rdb_fsync_range(fd,off,size) sync_file_range(fd,off,size,SYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITE)
#else
#define rdb_fsync_range(fd,off,size) fsync(fd)
#endif /* Check if we can use setproctitle().
* BSD systems have support for it, we provide an implementation for
* Linux and osx. */
#if (defined __NetBSD__ || defined __FreeBSD__ || defined __OpenBSD__)
#define USE_SETPROCTITLE
#endif #if ((defined __linux && defined(__GLIBC__)) || defined __APPLE__)
#define USE_SETPROCTITLE
#define INIT_SETPROCTITLE_REPLACEMENT
void spt_init(int argc, char *argv[]);
void setproctitle(const char *fmt, ...);
#endif /* Byte ordering detection */
#include <sys/types.h> /* This will likely define BYTE_ORDER */ #ifndef BYTE_ORDER
#if (BSD >= 199103)
# include <machine/endian.h>
#else
#if defined(linux) || defined(__linux__)
# include <endian.h>
#else
#define LITTLE_ENDIAN 1234 /* least-significant byte first (vax, pc) */
#define BIG_ENDIAN 4321 /* most-significant byte first (IBM, net) */
#define PDP_ENDIAN 3412 /* LSB first in word, MSW first in long (pdp)*/ #if defined(__i386__) || defined(__x86_64__) || defined(__amd64__) || \
defined(vax) || defined(ns32000) || defined(sun386) || \
defined(MIPSEL) || defined(_MIPSEL) || defined(BIT_ZERO_ON_RIGHT) || \
defined(__alpha__) || defined(__alpha)
#define BYTE_ORDER LITTLE_ENDIAN
#endif #if defined(sel) || defined(pyr) || defined(mc68000) || defined(sparc) || \
defined(is68k) || defined(tahoe) || defined(ibm032) || defined(ibm370) || \
defined(MIPSEB) || defined(_MIPSEB) || defined(_IBMR2) || defined(DGUX) ||\
defined(apollo) || defined(__convex__) || defined(_CRAY) || \
defined(__hppa) || defined(__hp9000) || \
defined(__hp9000s300) || defined(__hp9000s700) || \
defined (BIT_ZERO_ON_LEFT) || defined(m68k) || defined(__sparc)
#define BYTE_ORDER BIG_ENDIAN
#endif
#endif /* linux */
#endif /* BSD */
#endif /* BYTE_ORDER */ /* Sometimes after including an OS-specific header that defines the
* endianess we end with __BYTE_ORDER but not with BYTE_ORDER that is what
* the Redis code uses. In this case let's define everything without the
* underscores. */
#ifndef BYTE_ORDER
#ifdef __BYTE_ORDER
#if defined(__LITTLE_ENDIAN) && defined(__BIG_ENDIAN)
#ifndef LITTLE_ENDIAN
#define LITTLE_ENDIAN __LITTLE_ENDIAN
#endif
#ifndef BIG_ENDIAN
#define BIG_ENDIAN __BIG_ENDIAN
#endif
#if (__BYTE_ORDER == __LITTLE_ENDIAN)
#define BYTE_ORDER LITTLE_ENDIAN
#else
#define BYTE_ORDER BIG_ENDIAN
#endif
#endif
#endif
#endif #if !defined(BYTE_ORDER) || \
(BYTE_ORDER != BIG_ENDIAN && BYTE_ORDER != LITTLE_ENDIAN)
/* you must determine what the correct bit order is for
* your compiler - the next line is an intentional error
* which will force your compiles to bomb until you fix
* the above macros.
*/
#error "Undefined or invalid BYTE_ORDER"
#endif #if (__i386 || __amd64 || __powerpc__) && __GNUC__
#define GNUC_VERSION (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__)
#if defined(__clang__)
#define HAVE_ATOMIC
#endif
#if (defined(__GLIBC__) && defined(__GLIBC_PREREQ))
#if (GNUC_VERSION >= 40100 && __GLIBC_PREREQ(2, 6))
#define HAVE_ATOMIC
#endif
#endif
#endif /* Make sure we can test for ARM just checking for __arm__, since sometimes
* __arm is defined but __arm__ is not. */
#if defined(__arm) && !defined(__arm__)
#define __arm__
#endif
#if defined (__aarch64__) && !defined(__arm64__)
#define __arm64__
#endif /* Make sure we can test for SPARC just checking for __sparc__. */
#if defined(__sparc) && !defined(__sparc__)
#define __sparc__
#endif #if defined(__sparc__) || defined(__arm__)
#define USE_ALIGNED_ACCESS
#endif #endif
从宏定义中可以看出来, redis 依赖 linux unix 类型的操作系统. 如果当时 redis 一心只为
linux 服务, 预计开发和维护的心智负担会小很多(纯属意淫). 那开始扯皮吧.
2.1' 宏排版差评, 写起来辣眼睛
我们以 BYTE_ORDER 为例子, 不放给其排排版, 对对齐, 方便肉眼阅读.
// Sometimes after including an OS-specific header that defines the
// endianess we end with __BYTE_ORDER but not with BYTE_ORDER that is what
// the Redis code uses. In this case let's define everything without the
// underscores.
#ifndef BYTE_ORDER
# ifdef __BYTE_ORDER # if defined __LITTLE_ENDIAN && defined __BIG_ENDIAN # ifndef LITTLE_ENDIAN
# define LITTLE_ENGIAN __LITTLE_ENDIAN
# endif # ifndef BIG_ENDIAN
# define BIG_ENGIAN __BIG_ENGIAN
# endif # if __BYTE_ORDER == __LITTLE_ENGIAN
# define BYTE_ORDER LITTLE_ENGIAN
# else
# define BYTE_ORDER BIG_ENGIAN
# endif # endif # endif
#endif
大家看看这样, 是不是清爽了很多.
而对于 config.h 我们不继续展开 config.c 了, 因为项目运行起点的就是 config. 这要再深入下去
基本就 redis all in 了. 附赠聊聊边角料 setproctitle 设置进程标题的话题.
#if (defined __linux && defined __GLIBC__) || (defined __APPLE__) #define USE_SETPROCTITLE #define INIT_SETPROCTITLE_REPLACEMENT extern void spt_init(int argc, char * argv[]);
extern void setproctitle(const char * fmt, ...); #endif
3. redis setproctitle.c 分析
/* ==========================================================================
* setproctitle.c - Linux/Darwin setproctitle.
* --------------------------------------------------------------------------
* Copyright (C) 2010 William Ahern
* Copyright (C) 2013 Salvatore Sanfilippo
* Copyright (C) 2013 Stam He
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to permit
* persons to whom the Software is furnished to do so, subject to the
* following conditions:
*
* The above copyright notice and this permission notice shall be included
* in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
* OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
* NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
* OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
* USE OR OTHER DEALINGS IN THE SOFTWARE.
* ==========================================================================
*/
#ifndef _GNU_SOURCE
#define _GNU_SOURCE
#endif #include <stddef.h> /* NULL size_t */
#include <stdarg.h> /* va_list va_start va_end */
#include <stdlib.h> /* malloc(3) setenv(3) clearenv(3) setproctitle(3) getprogname(3) */
#include <stdio.h> /* vsnprintf(3) snprintf(3) */ #include <string.h> /* strlen(3) strchr(3) strdup(3) memset(3) memcpy(3) */ #include <errno.h> /* errno program_invocation_name program_invocation_short_name */ #if !defined(HAVE_SETPROCTITLE)
#if (defined __NetBSD__ || defined __FreeBSD__ || defined __OpenBSD__ || defined __DragonFly__)
#define HAVE_SETPROCTITLE 1
#else
#define HAVE_SETPROCTITLE 0
#endif
#endif #if !HAVE_SETPROCTITLE
#if (defined __linux || defined __APPLE__) extern char **environ; static struct {
/* original value */
const char *arg0; /* title space available */
char *base, *end; /* pointer to original nul character within base */
char *nul; _Bool reset;
int error;
} SPT; #ifndef SPT_MIN
#define SPT_MIN(a, b) (((a) < (b))? (a) : (b))
#endif static inline size_t spt_min(size_t a, size_t b) {
return SPT_MIN(a, b);
} /* spt_min() */ /*
* For discussion on the portability of the various methods, see
* http://lists.freebsd.org/pipermail/freebsd-stable/2008-June/043136.html
*/
static int spt_clearenv(void) {
#if __GLIBC__
clearenv(); return ;
#else
extern char **environ;
static char **tmp; if (!(tmp = malloc(sizeof *tmp)))
return errno; tmp[] = NULL;
environ = tmp; return ;
#endif
} /* spt_clearenv() */ static int spt_copyenv(char *oldenv[]) {
extern char **environ;
char *eq;
int i, error; if (environ != oldenv)
return ; if ((error = spt_clearenv()))
goto error; for (i = ; oldenv[i]; i++) {
if (!(eq = strchr(oldenv[i], '=')))
continue; *eq = '\0';
error = ( != setenv(oldenv[i], eq + , ))? errno : ;
*eq = '='; if (error)
goto error;
} return ;
error:
environ = oldenv; return error;
} /* spt_copyenv() */ static int spt_copyargs(int argc, char *argv[]) {
char *tmp;
int i; for (i = ; i < argc || (i >= argc && argv[i]); i++) {
if (!argv[i])
continue; if (!(tmp = strdup(argv[i])))
return errno; argv[i] = tmp;
} return ;
} /* spt_copyargs() */ void spt_init(int argc, char *argv[]) {
char **envp = environ;
char *base, *end, *nul, *tmp;
int i, error; if (!(base = argv[]))
return; nul = &base[strlen(base)];
end = nul + ; for (i = ; i < argc || (i >= argc && argv[i]); i++) {
if (!argv[i] || argv[i] < end)
continue; end = argv[i] + strlen(argv[i]) + ;
} for (i = ; envp[i]; i++) {
if (envp[i] < end)
continue; end = envp[i] + strlen(envp[i]) + ;
} if (!(SPT.arg0 = strdup(argv[])))
goto syerr; #if __GLIBC__
if (!(tmp = strdup(program_invocation_name)))
goto syerr; program_invocation_name = tmp; if (!(tmp = strdup(program_invocation_short_name)))
goto syerr; program_invocation_short_name = tmp;
#elif __APPLE__
if (!(tmp = strdup(getprogname())))
goto syerr; setprogname(tmp);
#endif if ((error = spt_copyenv(envp)))
goto error; if ((error = spt_copyargs(argc, argv)))
goto error; SPT.nul = nul;
SPT.base = base;
SPT.end = end; return;
syerr:
error = errno;
error:
SPT.error = error;
} /* spt_init() */ #ifndef SPT_MAXTITLE
#define SPT_MAXTITLE 255
#endif void setproctitle(const char *fmt, ...) {
char buf[SPT_MAXTITLE + ]; /* use buffer in case argv[0] is passed */
va_list ap;
char *nul;
int len, error; if (!SPT.base)
return; if (fmt) {
va_start(ap, fmt);
len = vsnprintf(buf, sizeof buf, fmt, ap);
va_end(ap);
} else {
len = snprintf(buf, sizeof buf, "%s", SPT.arg0);
} if (len <= )
{ error = errno; goto error; } if (!SPT.reset) {
memset(SPT.base, , SPT.end - SPT.base);
SPT.reset = ;
} else {
memset(SPT.base, , spt_min(sizeof buf, SPT.end - SPT.base));
} len = spt_min(len, spt_min(sizeof buf, SPT.end - SPT.base) - );
memcpy(SPT.base, buf, len);
nul = &SPT.base[len]; if (nul < SPT.nul) {
*SPT.nul = '.';
} else if (nul == SPT.nul && &nul[] < SPT.end) {
*SPT.nul = ' ';
*++nul = '\0';
} return;
error:
SPT.error = error;
} /* setproctitle() */ #endif /* __linux || __APPLE__ */
#endif /* !HAVE_SETPROCTITLE */
3.1' spt_clearenv -> spt_copyenv -> setenv -> goto error -> environ = oldenv memory leak
https://github.com/antirez/redis/pull/6588/commits/ec5405b7ccf809929ff105aeb14d9854896f9d68
感兴趣的朋友可以一块交流. 想了解更多也可以参阅我和这个博主之间的交互(设置进程名称)
https://www.cnblogs.com/imlgc/p/3823990.html#4428962
4. redis atomicvar.h 分析
/* This file implements atomic counters using __atomic or __sync macros if
* available, otherwise synchronizing different threads using a mutex.
*
* The exported interface is composed of three macros:
*
* atomicIncr(var,count) -- Increment the atomic counter
* atomicGetIncr(var,oldvalue_var,count) -- Get and increment the atomic counter
* atomicDecr(var,count) -- Decrement the atomic counter
* atomicGet(var,dstvar) -- Fetch the atomic counter value
* atomicSet(var,value) -- Set the atomic counter value
*
* The variable 'var' should also have a declared mutex with the same
* name and the "_mutex" postfix, for instance:
*
* long myvar;
* pthread_mutex_t myvar_mutex;
* atomicSet(myvar,12345);
*
* If atomic primitives are available (tested in config.h) the mutex
* is not used.
*
* Never use return value from the macros, instead use the AtomicGetIncr()
* if you need to get the current value and increment it atomically, like
* in the followign example:
*
* long oldvalue;
* atomicGetIncr(myvar,oldvalue,1);
* doSomethingWith(oldvalue);
*
* ----------------------------------------------------------------------------
*
* Copyright (c) 2015, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/ #include <pthread.h> #ifndef __ATOMIC_VAR_H
#define __ATOMIC_VAR_H /* To test Redis with Helgrind (a Valgrind tool) it is useful to define
* the following macro, so that __sync macros are used: those can be detected
* by Helgrind (even if they are less efficient) so that no false positive
* is reported. */
// #define __ATOMIC_VAR_FORCE_SYNC_MACROS #if !defined(__ATOMIC_VAR_FORCE_SYNC_MACROS) && defined(__ATOMIC_RELAXED) && !defined(__sun) && (!defined(__clang__) || !defined(__APPLE__) || __apple_build_version__ > 4210057)
/* Implementation using __atomic macros. */ #define atomicIncr(var,count) __atomic_add_fetch(&var,(count),__ATOMIC_RELAXED)
#define atomicGetIncr(var,oldvalue_var,count) do { \
oldvalue_var = __atomic_fetch_add(&var,(count),__ATOMIC_RELAXED); \
} while()
#define atomicDecr(var,count) __atomic_sub_fetch(&var,(count),__ATOMIC_RELAXED)
#define atomicGet(var,dstvar) do { \
dstvar = __atomic_load_n(&var,__ATOMIC_RELAXED); \
} while()
#define atomicSet(var,value) __atomic_store_n(&var,value,__ATOMIC_RELAXED)
#define REDIS_ATOMIC_API "atomic-builtin" #elif defined(HAVE_ATOMIC)
/* Implementation using __sync macros. */ #define atomicIncr(var,count) __sync_add_and_fetch(&var,(count))
#define atomicGetIncr(var,oldvalue_var,count) do { \
oldvalue_var = __sync_fetch_and_add(&var,(count)); \
} while()
#define atomicDecr(var,count) __sync_sub_and_fetch(&var,(count))
#define atomicGet(var,dstvar) do { \
dstvar = __sync_sub_and_fetch(&var,); \
} while()
#define atomicSet(var,value) do { \
while(!__sync_bool_compare_and_swap(&var,var,value)); \
} while()
#define REDIS_ATOMIC_API "sync-builtin" #else
/* Implementation using pthread mutex. */ #define atomicIncr(var,count) do { \
pthread_mutex_lock(&var ## _mutex); \
var += (count); \
pthread_mutex_unlock(&var ## _mutex); \
} while()
#define atomicGetIncr(var,oldvalue_var,count) do { \
pthread_mutex_lock(&var ## _mutex); \
oldvalue_var = var; \
var += (count); \
pthread_mutex_unlock(&var ## _mutex); \
} while()
#define atomicDecr(var,count) do { \
pthread_mutex_lock(&var ## _mutex); \
var -= (count); \
pthread_mutex_unlock(&var ## _mutex); \
} while()
#define atomicGet(var,dstvar) do { \
pthread_mutex_lock(&var ## _mutex); \
dstvar = var; \
pthread_mutex_unlock(&var ## _mutex); \
} while()
#define atomicSet(var,value) do { \
pthread_mutex_lock(&var ## _mutex); \
var = value; \
pthread_mutex_unlock(&var ## _mutex); \
} while()
#define REDIS_ATOMIC_API "pthread-mutex" #endif
#endif /* __ATOMIC_VAR_H */
atomicvar.h 原子库操作封装思路有三种, C11 stdatomic.h 和 GCC sync 操作, 还有 POSIX pthread.h . 不过使用
pthread.h 封装的"原子操作", 不是那么通用, 因为和业务强绑定. 只能在 redis 项目下用. 原因是它依赖事先定义
好变量 var##_mutex
pthread_mutex_lock(&var ## _mutex); \
可以找到例子, 例如 src/lazyfree.c 中有段代码如下
#include "server.h"
#include "bio.h"
#include "atomicvar.h"
#include "cluster.h" static size_t lazyfree_objects = ;
pthread_mutex_t lazyfree_objects_mutex = PTHREAD_MUTEX_INITIALIZER;
事先定义 lazyfree_objects 和 lazyfree_objects_mutex 才能使用 pthread 封装的原子操作宏. 额外的优化
可以通过 __sync_lock_test_and_set 替代 while __sync_bool_compare_and_swap 费力操作
https://github.com/antirez/redis/pull/6567/commits/dc1e369d6c12df73f128822b7ba30cdc4dd4357a
5. redis zmalloc 分析
/* zmalloc - total amount of allocated memory aware version of malloc()
*
* Copyright (c) 2009-2010, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/ #ifndef __ZMALLOC_H
#define __ZMALLOC_H /* Double expansion needed for stringification of macro values. */
#define __xstr(s) __str(s)
#define __str(s) #s #if defined(USE_TCMALLOC)
#define ZMALLOC_LIB ("tcmalloc-" __xstr(TC_VERSION_MAJOR) "." __xstr(TC_VERSION_MINOR))
#include <google/tcmalloc.h>
#if (TC_VERSION_MAJOR == 1 && TC_VERSION_MINOR >= 6) || (TC_VERSION_MAJOR > 1)
#define HAVE_MALLOC_SIZE 1
#define zmalloc_size(p) tc_malloc_size(p)
#else
#error "Newer version of tcmalloc required"
#endif #elif defined(USE_JEMALLOC)
#define ZMALLOC_LIB ("jemalloc-" __xstr(JEMALLOC_VERSION_MAJOR) "." __xstr(JEMALLOC_VERSION_MINOR) "." __xstr(JEMALLOC_VERSION_BUGFIX))
#include <jemalloc/jemalloc.h>
#if (JEMALLOC_VERSION_MAJOR == 2 && JEMALLOC_VERSION_MINOR >= 1) || (JEMALLOC_VERSION_MAJOR > 2)
#define HAVE_MALLOC_SIZE 1
#define zmalloc_size(p) je_malloc_usable_size(p)
#else
#error "Newer version of jemalloc required"
#endif #elif defined(__APPLE__)
#include <malloc/malloc.h>
#define HAVE_MALLOC_SIZE 1
#define zmalloc_size(p) malloc_size(p)
#endif #ifndef ZMALLOC_LIB
#define ZMALLOC_LIB "libc"
#ifdef __GLIBC__
#include <malloc.h>
#define HAVE_MALLOC_SIZE 1
#define zmalloc_size(p) malloc_usable_size(p)
#endif
#endif /* We can enable the Redis defrag capabilities only if we are using Jemalloc
* and the version used is our special version modified for Redis having
* the ability to return per-allocation fragmentation hints. */
#if defined(USE_JEMALLOC) && defined(JEMALLOC_FRAG_HINT)
#define HAVE_DEFRAG
#endif void *zmalloc(size_t size);
void *zcalloc(size_t size);
void *zrealloc(void *ptr, size_t size);
void zfree(void *ptr);
char *zstrdup(const char *s);
size_t zmalloc_used_memory(void);
void zmalloc_set_oom_handler(void (*oom_handler)(size_t));
size_t zmalloc_get_rss(void);
int zmalloc_get_allocator_info(size_t *allocated, size_t *active, size_t *resident);
void set_jemalloc_bg_thread(int enable);
int jemalloc_purge();
size_t zmalloc_get_private_dirty(long pid);
size_t zmalloc_get_smap_bytes_by_field(char *field, long pid);
size_t zmalloc_get_memory_size(void);
void zlibc_free(void *ptr); #ifdef HAVE_DEFRAG
void zfree_no_tcache(void *ptr);
void *zmalloc_no_tcache(size_t size);
#endif #ifndef HAVE_MALLOC_SIZE
size_t zmalloc_size(void *ptr);
size_t zmalloc_usable(void *ptr);
#else
#define zmalloc_usable(p) zmalloc_size(p)
#endif #ifdef REDIS_TEST
int zmalloc_test(int argc, char **argv);
#endif #endif /* __ZMALLOC_H */
内存模块支持外部库丰富, 自然写的就有点啰嗦. 我们这里有个诀窍, 我们假定只使用 USE_JEMALLOC ,
然后代码一路走下去, 是不是吼方便呢.
/* zmalloc - total amount of allocated memory aware version of malloc()
*
* Copyright (c) 2009-2010, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/ #include <stdio.h>
#include <stdlib.h>
#include <stdint.h> /* This function provide us access to the original libc free(). This is useful
* for instance to free results obtained by backtrace_symbols(). We need
* to define this function before including zmalloc.h that may shadow the
* free implementation if we use jemalloc or another non standard allocator. */
void zlibc_free(void *ptr) {
free(ptr);
} #include <string.h>
#include <pthread.h>
#include "config.h"
#include "zmalloc.h"
#include "atomicvar.h" #ifdef HAVE_MALLOC_SIZE
#define PREFIX_SIZE (0)
#else
#if defined(__sun) || defined(__sparc) || defined(__sparc__)
#define PREFIX_SIZE (sizeof(long long))
#else
#define PREFIX_SIZE (sizeof(size_t))
#endif
#endif /* Explicitly override malloc/free etc when using tcmalloc. */
#if defined(USE_TCMALLOC)
#define malloc(size) tc_malloc(size)
#define calloc(count,size) tc_calloc(count,size)
#define realloc(ptr,size) tc_realloc(ptr,size)
#define free(ptr) tc_free(ptr)
#elif defined(USE_JEMALLOC)
#define malloc(size) je_malloc(size)
#define calloc(count,size) je_calloc(count,size)
#define realloc(ptr,size) je_realloc(ptr,size)
#define free(ptr) je_free(ptr)
#define mallocx(size,flags) je_mallocx(size,flags)
#define dallocx(ptr,flags) je_dallocx(ptr,flags)
#endif #define update_zmalloc_stat_alloc(__n) do { \
size_t _n = (__n); \
if (_n&(sizeof(long)-)) _n += sizeof(long)-(_n&(sizeof(long)-)); \
atomicIncr(used_memory,__n); \
} while() #define update_zmalloc_stat_free(__n) do { \
size_t _n = (__n); \
if (_n&(sizeof(long)-)) _n += sizeof(long)-(_n&(sizeof(long)-)); \
atomicDecr(used_memory,__n); \
} while() static size_t used_memory = ;
pthread_mutex_t used_memory_mutex = PTHREAD_MUTEX_INITIALIZER; static void zmalloc_default_oom(size_t size) {
fprintf(stderr, "zmalloc: Out of memory trying to allocate %zu bytes\n",
size);
fflush(stderr);
abort();
} static void (*zmalloc_oom_handler)(size_t) = zmalloc_default_oom; void *zmalloc(size_t size) {
void *ptr = malloc(size+PREFIX_SIZE); if (!ptr) zmalloc_oom_handler(size);
#ifdef HAVE_MALLOC_SIZE
update_zmalloc_stat_alloc(zmalloc_size(ptr));
return ptr;
#else
*((size_t*)ptr) = size;
update_zmalloc_stat_alloc(size+PREFIX_SIZE);
return (char*)ptr+PREFIX_SIZE;
#endif
} /* Allocation and free functions that bypass the thread cache
* and go straight to the allocator arena bins.
* Currently implemented only for jemalloc. Used for online defragmentation. */
#ifdef HAVE_DEFRAG
void *zmalloc_no_tcache(size_t size) {
void *ptr = mallocx(size+PREFIX_SIZE, MALLOCX_TCACHE_NONE);
if (!ptr) zmalloc_oom_handler(size);
update_zmalloc_stat_alloc(zmalloc_size(ptr));
return ptr;
} void zfree_no_tcache(void *ptr) {
if (ptr == NULL) return;
update_zmalloc_stat_free(zmalloc_size(ptr));
dallocx(ptr, MALLOCX_TCACHE_NONE);
}
#endif void *zcalloc(size_t size) {
void *ptr = calloc(, size+PREFIX_SIZE); if (!ptr) zmalloc_oom_handler(size);
#ifdef HAVE_MALLOC_SIZE
update_zmalloc_stat_alloc(zmalloc_size(ptr));
return ptr;
#else
*((size_t*)ptr) = size;
update_zmalloc_stat_alloc(size+PREFIX_SIZE);
return (char*)ptr+PREFIX_SIZE;
#endif
} void *zrealloc(void *ptr, size_t size) {
#ifndef HAVE_MALLOC_SIZE
void *realptr;
#endif
size_t oldsize;
void *newptr; if (size == && ptr != NULL) {
zfree(ptr);
return NULL;
}
if (ptr == NULL) return zmalloc(size);
#ifdef HAVE_MALLOC_SIZE
oldsize = zmalloc_size(ptr);
newptr = realloc(ptr,size);
if (!newptr) zmalloc_oom_handler(size); update_zmalloc_stat_free(oldsize);
update_zmalloc_stat_alloc(zmalloc_size(newptr));
return newptr;
#else
realptr = (char*)ptr-PREFIX_SIZE;
oldsize = *((size_t*)realptr);
newptr = realloc(realptr,size+PREFIX_SIZE);
if (!newptr) zmalloc_oom_handler(size); *((size_t*)newptr) = size;
update_zmalloc_stat_free(oldsize+PREFIX_SIZE);
update_zmalloc_stat_alloc(size+PREFIX_SIZE);
return (char*)newptr+PREFIX_SIZE;
#endif
} /* Provide zmalloc_size() for systems where this function is not provided by
* malloc itself, given that in that case we store a header with this
* information as the first bytes of every allocation. */
#ifndef HAVE_MALLOC_SIZE
size_t zmalloc_size(void *ptr) {
void *realptr = (char*)ptr-PREFIX_SIZE;
size_t size = *((size_t*)realptr);
/* Assume at least that all the allocations are padded at sizeof(long) by
* the underlying allocator. */
if (size&(sizeof(long)-)) size += sizeof(long)-(size&(sizeof(long)-));
return size+PREFIX_SIZE;
}
size_t zmalloc_usable(void *ptr) {
return zmalloc_size(ptr)-PREFIX_SIZE;
}
#endif void zfree(void *ptr) {
#ifndef HAVE_MALLOC_SIZE
void *realptr;
size_t oldsize;
#endif if (ptr == NULL) return;
#ifdef HAVE_MALLOC_SIZE
update_zmalloc_stat_free(zmalloc_size(ptr));
free(ptr);
#else
realptr = (char*)ptr-PREFIX_SIZE;
oldsize = *((size_t*)realptr);
update_zmalloc_stat_free(oldsize+PREFIX_SIZE);
free(realptr);
#endif
} char *zstrdup(const char *s) {
size_t l = strlen(s)+;
char *p = zmalloc(l); memcpy(p,s,l);
return p;
} size_t zmalloc_used_memory(void) {
size_t um;
atomicGet(used_memory,um);
return um;
} void zmalloc_set_oom_handler(void (*oom_handler)(size_t)) {
zmalloc_oom_handler = oom_handler;
} /* Get the RSS information in an OS-specific way.
*
* WARNING: the function zmalloc_get_rss() is not designed to be fast
* and may not be called in the busy loops where Redis tries to release
* memory expiring or swapping out objects.
*
* For this kind of "fast RSS reporting" usages use instead the
* function RedisEstimateRSS() that is a much faster (and less precise)
* version of the function. */ #if defined(HAVE_PROC_STAT)
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h> size_t zmalloc_get_rss(void) {
int page = sysconf(_SC_PAGESIZE);
size_t rss;
char buf[];
char filename[];
int fd, count;
char *p, *x; snprintf(filename,,"/proc/%d/stat",getpid());
if ((fd = open(filename,O_RDONLY)) == -) return ;
if (read(fd,buf,) <= ) {
close(fd);
return ;
}
close(fd); p = buf;
count = ; /* RSS is the 24th field in /proc/<pid>/stat */
while(p && count--) {
p = strchr(p,' ');
if (p) p++;
}
if (!p) return ;
x = strchr(p,' ');
if (!x) return ;
*x = '\0'; rss = strtoll(p,NULL,);
rss *= page;
return rss;
}
#elif defined(HAVE_TASKINFO)
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/sysctl.h>
#include <mach/task.h>
#include <mach/mach_init.h> size_t zmalloc_get_rss(void) {
task_t task = MACH_PORT_NULL;
struct task_basic_info t_info;
mach_msg_type_number_t t_info_count = TASK_BASIC_INFO_COUNT; if (task_for_pid(current_task(), getpid(), &task) != KERN_SUCCESS)
return ;
task_info(task, TASK_BASIC_INFO, (task_info_t)&t_info, &t_info_count); return t_info.resident_size;
}
#elif defined(__FreeBSD__)
#include <sys/types.h>
#include <sys/sysctl.h>
#include <sys/user.h>
#include <unistd.h> size_t zmalloc_get_rss(void) {
struct kinfo_proc info;
size_t infolen = sizeof(info);
int mib[];
mib[] = CTL_KERN;
mib[] = KERN_PROC;
mib[] = KERN_PROC_PID;
mib[] = getpid(); if (sysctl(mib, , &info, &infolen, NULL, ) == )
return (size_t)info.ki_rssize; return 0L;
}
#else
size_t zmalloc_get_rss(void) {
/* If we can't get the RSS in an OS-specific way for this system just
* return the memory usage we estimated in zmalloc()..
*
* Fragmentation will appear to be always 1 (no fragmentation)
* of course... */
return zmalloc_used_memory();
}
#endif #if defined(USE_JEMALLOC) int zmalloc_get_allocator_info(size_t *allocated,
size_t *active,
size_t *resident) {
uint64_t epoch = ;
size_t sz;
*allocated = *resident = *active = ;
/* Update the statistics cached by mallctl. */
sz = sizeof(epoch);
je_mallctl("epoch", &epoch, &sz, &epoch, sz);
sz = sizeof(size_t);
/* Unlike RSS, this does not include RSS from shared libraries and other non
* heap mappings. */
je_mallctl("stats.resident", resident, &sz, NULL, );
/* Unlike resident, this doesn't not include the pages jemalloc reserves
* for re-use (purge will clean that). */
je_mallctl("stats.active", active, &sz, NULL, );
/* Unlike zmalloc_used_memory, this matches the stats.resident by taking
* into account all allocations done by this process (not only zmalloc). */
je_mallctl("stats.allocated", allocated, &sz, NULL, );
return ;
} void set_jemalloc_bg_thread(int enable) {
/* let jemalloc do purging asynchronously, required when there's no traffic
* after flushdb */
char val = !!enable;
je_mallctl("background_thread", NULL, , &val, );
} int jemalloc_purge() {
/* return all unused (reserved) pages to the OS */
char tmp[];
unsigned narenas = ;
size_t sz = sizeof(unsigned);
if (!je_mallctl("arenas.narenas", &narenas, &sz, NULL, )) {
sprintf(tmp, "arena.%d.purge", narenas);
if (!je_mallctl(tmp, NULL, , NULL, ))
return ;
}
return -;
} #else int zmalloc_get_allocator_info(size_t *allocated,
size_t *active,
size_t *resident) {
*allocated = *resident = *active = ;
return ;
} void set_jemalloc_bg_thread(int enable) {
((void)(enable));
} int jemalloc_purge() {
return ;
} #endif /* Get the sum of the specified field (converted form kb to bytes) in
* /proc/self/smaps. The field must be specified with trailing ":" as it
* apperas in the smaps output.
*
* If a pid is specified, the information is extracted for such a pid,
* otherwise if pid is -1 the information is reported is about the
* current process.
*
* Example: zmalloc_get_smap_bytes_by_field("Rss:",-1);
*/
#if defined(HAVE_PROC_SMAPS)
size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
char line[];
size_t bytes = ;
int flen = strlen(field);
FILE *fp; if (pid == -) {
fp = fopen("/proc/self/smaps","r");
} else {
char filename[];
snprintf(filename,sizeof(filename),"/proc/%ld/smaps",pid);
fp = fopen(filename,"r");
} if (!fp) return ;
while(fgets(line,sizeof(line),fp) != NULL) {
if (strncmp(line,field,flen) == ) {
char *p = strchr(line,'k');
if (p) {
*p = '\0';
bytes += strtol(line+flen,NULL,) * ;
}
}
}
fclose(fp);
return bytes;
}
#else
size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
((void) field);
((void) pid);
return ;
}
#endif size_t zmalloc_get_private_dirty(long pid) {
return zmalloc_get_smap_bytes_by_field("Private_Dirty:",pid);
} /* Returns the size of physical memory (RAM) in bytes.
* It looks ugly, but this is the cleanest way to achieve cross platform results.
* Cleaned up from:
*
* http://nadeausoftware.com/articles/2012/09/c_c_tip_how_get_physical_memory_size_system
*
* Note that this function:
* 1) Was released under the following CC attribution license:
* http://creativecommons.org/licenses/by/3.0/deed.en_US.
* 2) Was originally implemented by David Robert Nadeau.
* 3) Was modified for Redis by Matt Stancliff.
* 4) This note exists in order to comply with the original license.
*/
size_t zmalloc_get_memory_size(void) {
#if defined(__unix__) || defined(__unix) || defined(unix) || \
(defined(__APPLE__) && defined(__MACH__))
#if defined(CTL_HW) && (defined(HW_MEMSIZE) || defined(HW_PHYSMEM64))
int mib[];
mib[] = CTL_HW;
#if defined(HW_MEMSIZE)
mib[] = HW_MEMSIZE; /* OSX. --------------------- */
#elif defined(HW_PHYSMEM64)
mib[] = HW_PHYSMEM64; /* NetBSD, OpenBSD. --------- */
#endif
int64_t size = ; /* 64-bit */
size_t len = sizeof(size);
if (sysctl( mib, , &size, &len, NULL, ) == )
return (size_t)size;
return 0L; /* Failed? */ #elif defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE)
/* FreeBSD, Linux, OpenBSD, and Solaris. -------------------- */
return (size_t)sysconf(_SC_PHYS_PAGES) * (size_t)sysconf(_SC_PAGESIZE); #elif defined(CTL_HW) && (defined(HW_PHYSMEM) || defined(HW_REALMEM))
/* DragonFly BSD, FreeBSD, NetBSD, OpenBSD, and OSX. -------- */
int mib[];
mib[] = CTL_HW;
#if defined(HW_REALMEM)
mib[] = HW_REALMEM; /* FreeBSD. ----------------- */
#elif defined(HW_PHYSMEM)
mib[] = HW_PHYSMEM; /* Others. ------------------ */
#endif
unsigned int size = ; /* 32-bit */
size_t len = sizeof(size);
if (sysctl(mib, , &size, &len, NULL, ) == )
return (size_t)size;
return 0L; /* Failed? */
#else
return 0L; /* Unknown method to get the data. */
#endif
#else
return 0L; /* Unknown OS. */
#endif
} #ifdef REDIS_TEST
#define UNUSED(x) ((void)(x))
int zmalloc_test(int argc, char **argv) {
void *ptr; UNUSED(argc);
UNUSED(argv);
printf("Initial used memory: %zu\n", zmalloc_used_memory());
ptr = zmalloc();
printf("Allocated 123 bytes; used: %zu\n", zmalloc_used_memory());
ptr = zrealloc(ptr, );
printf("Reallocated to 456 bytes; used: %zu\n", zmalloc_used_memory());
zfree(ptr);
printf("Freed pointer; used: %zu\n", zmalloc_used_memory());
return ;
}
#endif
这里提了一个优化, 减少代码的无效操作.
https://github.com/antirez/redis/pull/6591/commits/ddeece5d690ec41af051a842afe20a3e0ba25d37
整体代码非常简单. PREFIX_SIZE 用于记录申请内存块大小用的, 额外的, 写写, 自然就全明白了. 相比其它
模块多了 zmalloc_test 单元测试. 调用的地方可以看 src/server.c
void _serverPanic(const char *file, int line, const char *msg, ...) {
va_list ap;
va_start(ap,msg);
char fmtmsg[];
vsnprintf(fmtmsg,sizeof(fmtmsg),msg,ap);
va_end(ap);
bugReportStart();
serverLog(LL_WARNING,"------------------------------------------------");
serverLog(LL_WARNING,"!!! Software Failure. Press left mouse button to continue");
serverLog(LL_WARNING,"Guru Meditation: %s #%s:%d",fmtmsg,file,line);
#ifdef HAVE_BACKTRACE
serverLog(LL_WARNING,"(forcing SIGSEGV in order to print the stack trace)");
#endif
serverLog(LL_WARNING,"------------------------------------------------");
*((char*)-) = 'x';
}
#define serverPanic(...) _serverPanic(__FILE__,__LINE__,__VA_ARGS__),_exit(1)
void redisOutOfMemoryHandler(size_t allocation_size) {
serverLog(LL_WARNING,"Out Of Memory allocating %zu bytes!",
allocation_size);
serverPanic("Redis aborting for OUT OF MEMORY");
}
void redisSetProcTitle(char *title) {
#ifdef USE_SETPROCTITLE
char *server_mode = "";
if (server.cluster_enabled) server_mode = " [cluster]";
else if (server.sentinel_mode) server_mode = " [sentinel]";
setproctitle("%s %s:%d%s",
title,
server.bindaddr_count ? server.bindaddr[] : "*",
server.port ? server.port : server.tls_port,
server_mode);
#else
UNUSED(title);
#endif
}
int main(int argc, char **argv) {
struct timeval tv;
int j;
#ifdef REDIS_TEST
if (argc == && !strcasecmp(argv[], "test")) {
if (!strcasecmp(argv[], "ziplist")) {
return ziplistTest(argc, argv);
} else if (!strcasecmp(argv[], "quicklist")) {
quicklistTest(argc, argv);
} else if (!strcasecmp(argv[], "intset")) {
return intsetTest(argc, argv);
} else if (!strcasecmp(argv[], "zipmap")) {
return zipmapTest(argc, argv);
} else if (!strcasecmp(argv[], "sha1test")) {
return sha1Test(argc, argv);
} else if (!strcasecmp(argv[], "util")) {
return utilTest(argc, argv);
} else if (!strcasecmp(argv[], "endianconv")) {
return endianconvTest(argc, argv);
} else if (!strcasecmp(argv[], "crc64")) {
return crc64Test(argc, argv);
} else if (!strcasecmp(argv[], "zmalloc")) {
return zmalloc_test(argc, argv);
}
return -; /* test not found */
}
#endif
/* We need to initialize our libraries, and the server configuration. */
#ifdef INIT_SETPROCTITLE_REPLACEMENT
spt_init(argc, argv);
#endif
setlocale(LC_COLLATE,"");
tzset(); /* Populates 'timezone' global. */
zmalloc_set_oom_handler(redisOutOfMemoryHandler);
srand(time(NULL)^getpid());
gettimeofday(&tv,NULL);
...
..
.
通过这些预计你也看出点门道了吧. 关于 adlist 代码应该也了解七七八八了. 最后我们进入操练编译环节,
打通边角最后的一公里.
6. redis Makefile 解析
加了注释, 方便大家自行逐个阅读 mkreleasehdr.sh 和 Makefile, 感悟编译的不容易.
#!/bin/sh # git log 第一个八位hash 值
GIT_SHA1=`(git show-ref --head --hash= >/dev/null || echo ) | head -n1` # 显示提交, 提交和工作树等之间的变化, 禁止外部差异驱动程序. 系统默认的 git diff
GIT_DIRTY=`git diff --no-ext-diff >/dev/null | wc -l` # {网络节点上的主机名}-{时间戳}
BUILD_ID=`uname -n`"-"`date +%s`
# -n 判断 "$SOURCE_DATE_EPOCH" 是否不是空串, 不是为真
if [ -n "$SOURCE_DATE_EPOCH" ] then
# SOURCE_DATE_EPOCH= -> date -u -d "@1574154953" +%s -> 还是传入的时间戳
# SOURCE_DATE_EPOCH=src -> date -u -r "src" +%s -> 显示指定文件的最后修改时间
# date -u +%s -> -> 输出或者设置协调的通用时间
BUILD_ID=$(date -u -d "@$SOURCE_DATE_EPOCH" +%s >/dev/null || date -u -r "$SOURCE_DATE_EPOCH" +%s >/dev/null || date -u +%s)
fi # 测试 release.h 文件是否存在, 不存在就创建
test -f release.h || touch release.h
# 判断 release.h 是否已经创建 OK, OK 的话 exit 退出
(cat release.h | grep SHA1 | grep $GIT_SHA1) && \
(cat release.h | grep DIRTY | grep $GIT_DIRTY) && exit # Already up-to-date # release.h 写入宏配置 REDIS_GIT_SHA1 REDIS_GIT_DIRTY REDIS_BUILD_ID
echo "#define REDIS_GIT_SHA1 \"$GIT_SHA1\"" > release.h
echo "#define REDIS_GIT_DIRTY \"$GIT_DIRTY\"" >> release.h
echo "#define REDIS_BUILD_ID \"$BUILD_ID\"" >> release.h # touch 更新 文件访问时间 atime 文件内容更改时间 mtime 文件状态改动时间 ctime
touch release.c # Force recompile release.c
# Redis Makefile
# Copyright (C) Salvatore Sanfilippo <antirez at gmail dot com>
# This file is released under the BSD license, see the COPYING file
#
# The Makefile composes the final FINAL_CFLAGS and FINAL_LDFLAGS using
# what is needed for Redis plus the standard CFLAGS and LDFLAGS passed.
# However when building the dependencies (Jemalloc, Lua, Hiredis, ...)
# CFLAGS and LDFLAGS are propagated to the dependencies, so to pass
# flags only to be used when compiling / linking Redis itself REDIS_CFLAGS
# and REDIS_LDFLAGS are used instead (this is the case of 'make gcov').
#
# Dependencies are stored in the Makefile.dep file. To rebuild this file
# Just use 'make dep', but this is only needed by developers. # Makefile run sh ./mkreleasehdr.sh
release_hdr := $(shell sh -c './mkreleasehdr.sh')
# 定义直接展示式变量 uname_S := 内核名称, 例如 uname -s -> Linux
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
# 定义直接展示式变量 uname_M := 主机的硬件架构名称, 例如 uname -s -> x86_64
uname_M := $(shell sh -c 'uname -m 2>/dev/null || echo not')
# 定义条件赋值变量, 当 OPTIMIZATION 不存在, 会有 OPTIMIZATION :=-O2, 否则不改变 OPTIMIZATION
OPTIMIZATION?=-O2
# 定义递归展开式变量 DEPENDENCY_TARGETS =hiredis linenoise lua
# Makefile 展开时候 = 后面如果有空格, 会保留.
DEPENDENCY_TARGETS=hiredis linenoise lua
# 定义直接展示式变量 NODEPS :=clean distclean
NODEPS:=clean distclean # Default settings
# -std=c11 使用 C11 标准
# -pedantic 以ANSI/ISO C标准列出的所有警告
# -DREDIS_STATIC='' 进行 REDIS_STATIC='' 宏定义
STD=-std=c11 -pedantic -DREDIS_STATIC=''
# ifneq ($, $) $ != $ 为真
# $(findstring clang,$(CC)) 在 $(CC) 中查找 clang 字符串, 找到了返回 clang, 否则返回空串
# $(findstring FreeBSD,$(uname_S)) 在 $(uname_S) 中查找 FreeBSD 字符串 ...
# -Wno-c11-extensions 删除 C11 扩展警告
ifneq (,$(findstring clang,$(CC)))
ifneq (,$(findstring FreeBSD,$(uname_S)))
STD+=-Wno-c11-extensions
endif
endif
# -Wall 显示编译后所有警告
# -W 类似-Wall 会显示警告, 但是只显示编译器认为会出现错误的警告
# -Wno-missing-field-initializers 禁止结构初始化未指定所有字段警告
WARN=-Wall -W -Wno-missing-field-initializers
# 定义递归展开式变量 OPT =$(OPTIMIZATION) ?=-O2
OPT=$(OPTIMIZATION) # # 定义条件赋值变量, 当 PREFIX 不存在, 会有 PREFIX :=/usr/local, 否则维持 PREFIX 变量不变
PREFIX?=/usr/local
INSTALL_BIN=$(PREFIX)/bin
INSTALL=install # Default allocator defaults to Jemalloc if it's not an ARM
MALLOC=libc
# ifneq ($(uname_M),armv6l) $(uname_M) 硬件架构不是 armv6l 才为真
# ifneq ($(uname_M),armv7l) $(uname_M) 硬件架构不是 armv7l 才为真
# ifeq ($(uname_S),Linux) $(uname_S) 内核名称是 Linux 才为真
ifneq ($(uname_M),armv6l)
ifneq ($(uname_M),armv7l)
ifeq ($(uname_S),Linux)
MALLOC=jemalloc
endif
endif
endif # To get ARM stack traces if Redis crashes we need a special C flag.
# $(filter aarch64 armv,$(uname_M)) $(uname_M) 硬件架构中如果是 aarch64 or armv 就保留
# -funwind-tables 开启便于做 backtrace
# unwind table 表记录了与函数相关的信息, 函数的起始地址, 函数的结束地址, 一个 info block 指针
ifneq (,$(filter aarch64 armv,$(uname_M)))
CFLAGS+=-funwind-tables
else
ifneq (,$(findstring armv,$(uname_M)))
CFLAGS+=-funwind-tables
endif
endif # Backwards compatibility for selecting an allocator
ifeq ($(USE_TCMALLOC),yes)
MALLOC=tcmalloc
endif ifeq ($(USE_TCMALLOC_MINIMAL),yes)
MALLOC=tcmalloc_minimal
endif # ifeq ($(USE_JEMALLOC),yes) $(USE_JEMALLOC) == yes 会让 MALLOC=jemalloc
ifeq ($(USE_JEMALLOC),yes)
MALLOC=jemalloc
endif ifeq ($(USE_JEMALLOC),no)
MALLOC=libc
endif # Override default settings if possible
# - 忽略错误
# include .make-settings 导入 .make-settings Makefile 文件
-include .make-settings # 以 Linux jemalloc 为例
# FINAL_CFLAGS=-std=c11 -pedantic -DREDIS_STATIC='' -Wall -W -Wno-missing-field-initializers -O2 -g -ggdb
# FINAL_LDFLAGS= -g -ggdb
# -g 操作系统的原生格式(native format)生成调试信息, 其它调试器也可以用
# -ggdb 使 GCC 为 GDB 生成专用的更为丰富的调试信息
# -lm 链接 math.h 相关库
FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
FINAL_LDFLAGS=$(LDFLAGS) $(REDIS_LDFLAGS) $(DEBUG)
FINAL_LIBS=-lm
DEBUG=-g -ggdb ifeq ($(uname_S),SunOS)
# SunOS
ifneq ($(@@),32bit)
CFLAGS+= -m64
LDFLAGS+= -m64
endif
DEBUG=-g
DEBUG_FLAGS=-g
export CFLAGS LDFLAGS DEBUG DEBUG_FLAGS
INSTALL=cp -pf
FINAL_CFLAGS+= -D__EXTENSIONS__ -D_XPG6
FINAL_LIBS+= -ldl -lnsl -lsocket -lresolv -lpthread -lrt
else
ifeq ($(uname_S),Darwin)
# Darwin
FINAL_LIBS+= -ldl
OPENSSL_CFLAGS=-I/usr/local/opt/openssl/include
OPENSSL_LDFLAGS=-L/usr/local/opt/openssl/lib
else
ifeq ($(uname_S),AIX)
# AIX
FINAL_LDFLAGS+= -Wl,-bexpall
FINAL_LIBS+=-ldl -pthread -lcrypt -lbsd
else
ifeq ($(uname_S),OpenBSD)
# OpenBSD
FINAL_LIBS+= -lpthread
ifeq ($(USE_BACKTRACE),yes)
FINAL_CFLAGS+= -DUSE_BACKTRACE -I/usr/local/include
FINAL_LDFLAGS+= -L/usr/local/lib
FINAL_LIBS+= -lexecinfo
endif else
ifeq ($(uname_S),FreeBSD)
# FreeBSD
FINAL_LIBS+= -lpthread -lexecinfo
else
ifeq ($(uname_S),DragonFly)
# FreeBSD
FINAL_LIBS+= -lpthread -lexecinfo
else
# All the other OSes (notably Linux)
FINAL_LDFLAGS+= -rdynamic
FINAL_LIBS+=-ldl -pthread -lrt
endif
endif
endif
endif
endif
endif
# Include paths to dependencies
FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src ifeq ($(MALLOC),tcmalloc)
FINAL_CFLAGS+= -DUSE_TCMALLOC
FINAL_LIBS+= -ltcmalloc
endif ifeq ($(MALLOC),tcmalloc_minimal)
FINAL_CFLAGS+= -DUSE_TCMALLOC
FINAL_LIBS+= -ltcmalloc_minimal
endif # ifeq ($(MALLOC),jemalloc) $(MALLOC) == jemalloc 为真才进入条件分支
# DEPENDENCY_TARGETS += jemalloc -> DEPENDENCY_TARGETS = hiredis linenoise lua jemalloc
# FINAL_CFLAGS 继续 += -DUSE_JEMALLOC -I../deps/jemalloc/include 引入 USE_JEMALLOC 宏和指定头文件路径
# FINAL_LIBS := ../deps/jemalloc/lib/libjemalloc.a -lm
ifeq ($(MALLOC),jemalloc)
DEPENDENCY_TARGETS+= jemalloc
FINAL_CFLAGS+= -DUSE_JEMALLOC -I../deps/jemalloc/include
FINAL_LIBS := ../deps/jemalloc/lib/libjemalloc.a $(FINAL_LIBS)
endif # BUILD_TLS -> USE_OPENSSL
ifeq ($(BUILD_TLS),yes)
FINAL_CFLAGS+=-DUSE_OPENSSL $(OPENSSL_CFLAGS)
FINAL_LDFLAGS+=$(OPENSSL_LDFLAGS)
FINAL_LIBS += ../deps/hiredis/libhiredis_ssl.a -lssl -lcrypto
endif REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)
REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS)
REDIS_INSTALL=$(QUIET_INSTALL)$(INSTALL) # CCCOLOR="\033[34m" 蓝色字
# LINKCOLOR="\033[34;1m" 蓝色字 高亮
# SRCCOLOR="\033[33m" 黄色字
# BINCOLOR="\033[37;1m" 白色字 高亮
# MAKECOLOR="\033[32;1m" 绿色字 高亮
# ENDCOLOR="\033[0m" 关闭所有属性
CCCOLOR="\033[34m"
LINKCOLOR="\033[34;1m"
SRCCOLOR="\033[33m"
BINCOLOR="\033[37;1m"
MAKECOLOR="\033[32;1m"
ENDCOLOR="\033[0m" # ifndef 不存在 V 那么就为真
# @printf Makefile 屏幕上不再回显执行的命令
# %b 相对应的参数被视为含有要被处理的转义序列之字符串
# >& 标准输出重定向到标准错误
# $@ 表示规则中的目标
# QUIET_CC 编译, QUIET_LINK 链接, QUIET_INSTALL 安装
ifndef V
QUIET_CC = @printf ' %b %b\n' $(CCCOLOR)CC$(ENDCOLOR) $(SRCCOLOR)$@$(ENDCOLOR) >&;
QUIET_LINK = @printf ' %b %b\n' $(LINKCOLOR)LINK$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) >&;
QUIET_INSTALL = @printf ' %b %b\n' $(LINKCOLOR)INSTALL$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) >&;
endif REDIS_SERVER_NAME=redis-server
REDIS_SENTINEL_NAME=redis-sentinel
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o
REDIS_CLI_NAME=redis-cli
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o siphash.o crc16.o
REDIS_BENCHMARK_NAME=redis-benchmark
REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o siphash.o redis-benchmark.o
REDIS_CHECK_RDB_NAME=redis-check-rdb
REDIS_CHECK_AOF_NAME=redis-check-aof all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME)
@echo ""
@echo "Hint: It's a good idea to run 'make test' ;)"
@echo "" # 构建 Makefile.dep 文件
# -M 不是输出预编译过程的结果, 而是输出一个用于make的规则, 该规则描述了这个源文件的依赖关系.
# 预编译器输出的这个 make 规则包含名字与原文件相同的目标文件, 冒号和所有 include 文件的名字
# -MM 与 -M 相似, 只是不包含系统头文件
Makefile.dep:
-$(REDIS_CC) -MM *.c > Makefile.dep > /dev/null || true # $(words <text>) 单词个数统计函数
# MAKECMDGOALS 会存放你所指定的终极目标的列表，如果在命令行上，你没有指定目标，那么，这个变量是空值
# NODEPS:=clean distclean
ifeq (, $(words $(findstring $(MAKECMDGOALS), $(NODEPS))))
-include Makefile.dep
endif .PHONY: all # 构建 .make-settings
persist-settings: distclean
echo STD=$(STD) >> .make-settings
echo WARN=$(WARN) >> .make-settings
echo OPT=$(OPT) >> .make-settings
echo MALLOC=$(MALLOC) >> .make-settings
echo CFLAGS=$(CFLAGS) >> .make-settings
echo LDFLAGS=$(LDFLAGS) >> .make-settings
echo REDIS_CFLAGS=$(REDIS_CFLAGS) >> .make-settings
echo REDIS_LDFLAGS=$(REDIS_LDFLAGS) >> .make-settings
echo PREV_FINAL_CFLAGS=$(FINAL_CFLAGS) >> .make-settings
echo PREV_FINAL_LDFLAGS=$(FINAL_LDFLAGS) >> .make-settings
-(cd ../deps && $(MAKE) $(DEPENDENCY_TARGETS)) .PHONY: persist-settings # Prerequisites target
.make-prerequisites:
@touch $@ # Clean everything, persist settings and build dependencies if anything changed
ifneq ($(strip $(PREV_FINAL_CFLAGS)), $(strip $(FINAL_CFLAGS)))
.make-prerequisites: persist-settings
endif # $(strip STRINT) 去空格函数
ifneq ($(strip $(PREV_FINAL_LDFLAGS)), $(strip $(FINAL_LDFLAGS)))
.make-prerequisites: persist-settings
endif # redis-server
$(REDIS_SERVER_NAME): $(REDIS_SERVER_OBJ)
$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/lua/src/liblua.a $(FINAL_LIBS) # redis-sentinel
$(REDIS_SENTINEL_NAME): $(REDIS_SERVER_NAME)
$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) # redis-check-rdb
$(REDIS_CHECK_RDB_NAME): $(REDIS_SERVER_NAME)
$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_CHECK_RDB_NAME) # redis-check-aof
$(REDIS_CHECK_AOF_NAME): $(REDIS_SERVER_NAME)
$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_CHECK_AOF_NAME) # redis-cli
$(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/linenoise/linenoise.o $(FINAL_LIBS) # redis-benchmark
$(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a $(FINAL_LIBS) dict-benchmark: dict.c zmalloc.c sds.c siphash.c
$(REDIS_CC) $(FINAL_CFLAGS) $^ -D DICT_BENCHMARK_MAIN -o $@ $(FINAL_LIBS) # Because the jemalloc.h header is generated as a part of the jemalloc build,
# building it should complete before building any other object. Instead of
# depending on a single artifact, build all dependencies first.
%.o: %.c .make-prerequisites
$(REDIS_CC) -c $< clean:
rm -rf $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) *.o *.gcda *.gcno *.gcov redis.info lcov-html Makefile.dep dict-benchmark .PHONY: clean distclean: clean
-(cd ../deps && $(MAKE) distclean)
-(rm -f .make-*) .PHONY: distclean test: $(REDIS_SERVER_NAME) $(REDIS_CHECK_AOF_NAME)
@(cd ..; ./runtest) test-sentinel: $(REDIS_SENTINEL_NAME)
@(cd ..; ./runtest-sentinel) check: test lcov:
$(MAKE) gcov
@(set -e; cd ..; ./runtest --clients )
@geninfo -o redis.info .
@genhtml --legend -o lcov-html redis.info test-sds: sds.c sds.h
$(REDIS_CC) sds.c zmalloc.c -DSDS_TEST_MAIN $(FINAL_LIBS) -o /tmp/sds_test
/tmp/sds_test .PHONY: lcov bench: $(REDIS_BENCHMARK_NAME)
./$(REDIS_BENCHMARK_NAME) 32bit:
@echo ""
@echo "WARNING: if it fails under Linux you probably need to install libc6-dev-i386"
@echo ""
$(MAKE) CFLAGS="-m32" LDFLAGS="-m32" gcov:
$(MAKE) REDIS_CFLAGS="-fprofile-arcs -ftest-coverage -DCOVERAGE_TEST" REDIS_LDFLAGS="-fprofile-arcs -ftest-coverage" noopt:
$(MAKE) OPTIMIZATION="-O0" valgrind:
$(MAKE) OPTIMIZATION="-O0" MALLOC="libc" helgrind:
$(MAKE) OPTIMIZATION="-O0" MALLOC="libc" CFLAGS="-D__ATOMIC_VAR_FORCE_SYNC_MACROS" # 生成 src/help.h 头文件
# @../utils/generate-command-help.rb > help.h 通过 ruby 脚本请求
# https://raw.githubusercontent.com/antirez/redis-doc/master/commands.json 构建相关 C 中全局区变量信息
src/help.h:
@../utils/generate-command-help.rb > help.h install: all
@mkdir -p $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_BENCHMARK_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CHECK_RDB_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CHECK_AOF_NAME) $(INSTALL_BIN)
@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_SENTINEL_NAME) uninstall:
rm -f $(INSTALL_BIN)/{$(REDIS_SERVER_NAME),$(REDIS_BENCHMARK_NAME),$(REDIS_CLI_NAME),$(REDIS_CHECK_RDB_NAME),$(REDIS_CHECK_AOF_NAME),$(REDIS_SENTINEL_NAME)}
感谢大家 ☆ redis 最简单的数据结构 adlist 双向链表就带大家写到这了. 有缘再见 ~
后记 - 如果
错误是难免, 欢迎大家指正交流, 共同提高 ~
IF《如果》——Rudyard Kipling（拉迪亚德.吉普林）
If you can keep your head when all about you
Are losing theirs and blaming it on you;
If you can trust yourself when all men doubt you,
But make allowance for their doubting too;
If you can wait and not be tired by waiting,
Or, being lied about, don't deal in lies,
Or, being hated, don't give way to hating,
And yet don't look too good, nor talk too wise;
如果所有人都失去理智，咒骂你，
你仍能保持头脑清醒；
如果所有人都怀疑你，
你仍能坚信自己，让所有的怀疑动摇；
如果你要等待，不要因此厌烦，
为人所骗，不要因此骗人，
为人所恨，不要因此抱恨，
不要太乐观，不要自以为是；
If you can dream - and not make dreams your master;
If you can think - and not make thoughts your aim;
If you can meet with triumph and disaster
And treat those two imposters just the same;
If you can bear to hear the truth you've spoken
Twisted by knaves to make a trap for fools,
Or watch the things you gave your life to broken,
And stoop and build ‘em up with worn-out tools;
如果你是个追梦人——不要被梦主宰；
如果你是个爱思考的人——不要以思想者自居；
如果你遇到骄傲和挫折
把两者当骗子看待；
如果你能忍受，你曾讲过的事实
被恶棍扭曲，用于蒙骗傻子；
或者，看着你用毕生去看护的东西被破坏，
俯下身去，用破旧的工具把它修补；
If you can make one heap of all your winnings
And risk it on one turn of pitch-and-toss,
And lose, and start again at your beginnings
And never breath a word about your loss;
If you can force your heart and nerve and sinew
To serve your turn long after they are gone,
And so hold on when there is nothing in you
Except the Will which says to them: "Hold on";
如果在你赢得无数桂冠之后，
然后孤注一掷再搏一次，
失败过后，东山再起，
不要抱怨你的失败；
如果你能迫使自己，
在别人走后，长久坚守阵地，
在你心中已空荡荡无一物，
只有意志告诉你“坚持！”；
If you can talk with crowds and keep your virtue,
Or walk with kings - nor lose the common touch;
If neither foes nor loving friends can hurt you;
If all men count with you, but none too much;
If you can fill the unforgiving minute
With sixty seconds' worth of distance run -
Yours is the Earth and everything that's in it,
如果你与人交谈，能保持风度，
伴王同行，能保持距离；
如果仇敌和好友都不害你；
如果所有人都指望你，却无人全心全意；
如果你花六十秒进行短程跑，
填满那不可饶恕的一分钟——
你就可以拥有一个世界，
这个世界的一切都是你的，
更重要的是，孩子，你是个顶天立地的人。
