NoteGenerally, consistency model define the behavior how mutliple concurrent workflows read/write data or replciation. And it works similiar among the following situations: Mutilple threads read/write memory within a process Mutilple processes read/write shared memory within a machine Mutilple nodes read/write shared data within a cluster In the following paragraph, I’d like to use thread to stands for the concurrent workflow to ease the demostration. So you can simply repl ...

WarmUpBasic operations in GC Mark: mark the reachable objects Sweep: clear the unreachable objects Copy-Move: Copy the reachable objects to another space Compaction: Compact the reachable objects in-place Some GC docs regard the Copy-Move operation as Compaction as well. Parallel vs Concurrent Parallel: GC process runs multiple threads in parallel with STW. Concurrent: GC runs concurrently with the application without STW. Why STW? - GC racesGC races with Application: Change object referenc ...

What’s CacheA cache is a high-speed data storage layer that stores a subset of data, typically transient in nature, so that future requests for that data are served up faster than is possible by accessing the data’s primary storage location. ExampleCache is a general technique that uses any high-speed temporary storage to avoid accessing low-speed storage. It’s widely used in various situations: (CPUCache > RAM): CPU Cache saves some data to avoid accessing RAM. (RAM > DISK): RAM could be ...

What’s distributed transaction?The article describes some common algorithms to achieve the ACID transaction in Distributed System. Please review the following pages first if you are not familiar with some basic concepts, such as ACID, CAP, BASE. Database Transaction - ACID & Isolation Level Consistency in Distributed System - CAP、BASE The distributed transaction aims to achieve ACID among multiple databases. In a distributed transaction environment, multiple processes participate in a tr ...

ACIDACID is a set of properties of database transactions intended to guarantee data validity despite concurrency, errors, power failures, and other mishaps. A - Atomcity - 原子性ALL-or-NothingTransactions are often composed of multiple statements. Atomicity guarantees that each transaction is treated as a single “unit”, which either succeeds completely or fails completely: if any of the statements constituting a transaction fails to complete, the entire transaction fails and rollback the data to t ...

阅读本文之前建议对LSM树有一定的认识。本文将介绍LSM Tree的主流实现，即LevelDB和RocksDB作为KV数据库。 两者对外提供的主要接口基本一致，就是包含以下5个基本操作 get(K) 查找key K对应的value put(K,V) 插入键值对（K，V） update(K,V) 查找key K对应的value更新为V delete(K) 删除key K对应的条目 scan(K1,K2) 得到从K1到K2的所有key和value 我们将从静态和动态角度去介绍两个数据库先从LevelDB开始，相对好理解。毕竟RocksDB是在levelDB上做的改进 LeveldDB接下从静态和动态角度去介绍 静态视角：假想整个系统正在运行过程中（不断插入删除读取数据），此时我们给LevelDb照相，从照片可以看到之前系统的数据在内存和磁盘中是如何分布的，处于什么状态等.动态视角：了解系统是如何写入一条记录，读出一条记录，删除一条记录的，同时也包括除了这些接口操作外的内部操作比如compaction，系统运行时崩溃后如何恢复系统等等方面。 架构 - 静态视角 内存 （对应LSM的C ...

加密算法简述所谓加密算法就是指将信息变成密文的计算方法。 可以很简单也可以很复杂。有的加密算法就是对信息进行简单的替换和乱序， 这种加密算法最明显的缺陷就是算法本身必须保证是保密的。现代加密算法通常需要密匙来完成对信息的加密运算， 算法本身是可以公开的，只要保证密匙的安全就能保存信息的安全。 基于密匙的加密算法可以分为两大类: 对称加密算法：加密数据和解密数据都是用的同一个密匙。缺点是密钥不安全。如AES 非对称加密算法:密匙分为公钥和私钥，公钥大多数情况下用来加密数据， 私钥大多数情况下用来解密数据； 不能从公钥推导出私钥； 任何人都可以拥有公钥，都可以用来加密数据， 只有拥有私钥的人才能将信息解密。安全,但是速度比对称加密慢. PKI 公开密匙体相关概念 PKI：Public Key Infrastructure，公钥基础设施。不是协议,只是一个比较泛的体系基础概念. CA：Certificate Of Authority，认证中心。 RSA: 当今使用最为广泛的非对称加密算法。 数字证书：提供了一种发布公钥的简便途径； 一个数字证书包括：拥有者身份信息、公钥、CA数字签名、 ...

CAP Theorem常见的解释是三者最多只能两者相互满足， C-Consistency: All nodes see the same data at the same time. A-Availability: Reads and writes always succeed P-Partition tolerance: The system continues to operate despite arbitrary message loss or failure of part of the system CA: Single site cluster, therefore all nodes are always in contact, when a partition occurs, the system blocks. Choose C and A with compromising of P (Partition Tolerance). e.g. type of applications: Banking and Finance application, a s ...

Linux IO ModelBuffered IO (default)`alias: normal IO ReadA read process could be divided into 2 stages: Waiting for the data (from disk or network) to be ready in kernel page cache ( load disk data through DMA ) Copying the data from the kernel to the process WriteWrite back Model. Direct IOReadOnly 1 stage: loading data into process space by DMA Writedirect write to disk. Comparsion Buffered beats Direct decoupling disk and process reduce IO reads Direct beats Buffered self-caching appl ...

Compliation processProcess Graph General Description General Compilation (1,2,3) translates source code file to machine code file respectively leaves undefined functions/symbols to be filled in by linker Link links the object code with the library code to produce an executable file Specific Description Preprocessing -E Removal of Comments, Expansion of Macros, Expansion of the included files. The lines in our code that begin with the “#” character are preprocessor directives. C ...