| Node (deno) | Go | Python | Java | |
|---|---|---|---|---|
| Multi-Threads | ✔️ | ✔️ | ✔️ | ✔️ |
| When to use |
- Back End Technology
- Architecture knowledge
- Node
- Go language
- Python
- Java
- RESTful API
- Authentication & authorization
- Lua
- Microservices Architecture 微服务
- Load/Stree Test
- Theory fo distributed system
- Database
- Caching
- Desgin Pattern 23 种设计模式
- FMEA(Failure mode and effects analysis) 故障模式与影响分析
- Search
- File System
- Network
- Middleware
architect-awesome Full Stack Interview Questions & Answers
| Node | Article |
|---|---|
| node 技巧 | NodeJs 静态服务器 |
| Reference | Reference |
| Node.js 包教不包会 | 七天学会 NodeJS |
| 从零开始 nodejs 系列文章 | Node 入门 |
| Node 初学者入门，一本全面的 NodeJS 教程 |
next generation web framework for node.js Koa
| Article | Article |
|---|---|
| Go Books | |
| awesome-go | |
| Go kit |
Celery is a task queue with focus on real-time processing, while also supporting task scheduling.
Why we need Task Queue in Web Context?
These are some common use cases:
- Running something in the background. For example, to finish the web request as soon as possible, then update the users page incrementally. This gives the user the impression of good performance and “snappiness”, even though the real work might actually take some time.
- Running something after the web request has finished.
- Making sure something is done, by executing it asynchronously and using retries.
- Scheduling periodic work.
| tool |
|---|
| virtualenv |
Netlfix Eureka is a REST (Representational State Transfer) based service that is primarily used in the AWS cloud for locating services for the purpose of load balancing and failover of middle-tier servers
spring cloud netflix Service Discovery (Eureka), Circuit Breaker (Hystrix), Intelligent Routing (Zuul) and Client Side Load Balancing (Ribbon)
Sample Rest.js and Spring Data REST
Playframework is based on a lightweight, stateless, web-friendly architecture |tool| Maven is a tool that can now be used for building and managing any Java-based project. Google guava is a set of core libraries that includes new collection types (such as multimap and multiset), immutable collections, a graph library, functional types, an in-memory cache, and APIs/utilities for concurrency, I/O, hashing, primitives, reflection, string processing, and much more!
Netlfix Hystrix 限流熔断，
Alibaba Java Coding Guidelines pmd implements and IDE plugin
Learning Java Spring Framework Transaction Management 分布式事务 TCC(Try-Confirm-Cancel) Transaction
Spring boot admin, Admin UI for administration of spring boot applications
Best practies for a pragmatic RESTful API
HATEOAS ( Hypermedia as Engine of Application State)
Microsoft RESTful API guidelines
API Design Tool
Standard JSON API response format ?
SAML SAML Security Assertion Markup Language is an open standard that allows identity providers (IdP) to pass authorization credentials to service providers (SP).
OAuth OAuth 2 OAuth2 single page application
SSO how does single-sign-on work
CAS (Central Authentication Service)
CAS SSO CAS Install tutorial video CAS War Overlayer
Basic
tokn
JWT(JSON Web Token) https://jwt.io/
OpenID https://openid.net
Spring Security Domain Object Security (ACLs) https://www.concretepage.com/spring/spring-security/
LDAP OpenLDAP LDAP is often used by organizations as a central repository for user information and as an authentication service. It can also be used to store the role information for application users.
IAM (identity and access management) all forms of access control can ultimately be mapped back to one of four classic models:
- Discretionary Access Control (DAC),
- Mandatory Access Control (MAC),
- Role-based Access Control (RBAC),
- Attribute-based Access Control (ABAC)
http://blog.identityautomation.com/rbac-vs-abac-access-control-models-iam-explained
| Article |
|---|
| Four Authentication & authorization |
| Entity-level access restriction in the microservice architecture based on user or group membership |
| OpenSource |
|---|
| Keycloak an open source Identity and Access Management solution |
| An OpenID Connect reference implementation in Java on the Spring platform |
OAuth2 vs OpenID Connect vs SAML
authentication-and-authorization-in-microservices
kong基于 OpenResty 的 API 网关服务和网关服务管理层. Kong dashboard
Tools for Microservices
| 名称 | 说明 |
|---|---|
| API Fortress | API 测试和健康检测工具，为企业级 API 提供自动化的功能测试、健康检测和负载测试。它的设计原则是无代码，完全基于现代 API 架构实践和模式而构建 收费 |
| Graylog | Security Incident and Event Management Report |
| Istio | 它支持在 Kubernetes 上进行服务部署，其服务网格技术为微服务通信带来了可靠性、安全性和可管理性。其中，服务网格技术可以用于改善应用程序和微服务之间的关系和交互。开源 |
| Conductor | Netflix 的微服务编排引擎，是 Netflix OSS 生态系统的一部分。它可以运行在云端，并实现了微服务的流程编配。它还能够用于控制和可视化微服务之间的交互。 |
| Seneca | 开发人员可以使用 Seneca，来轻松构建基于消息的微服务，它是 Node.js 的微服务工具包，可以用它写出干净而且结构良好的代码，并系统化应用程序的业务逻辑。 |
| Elixir | Elixir is a dynamic, functional language designed for building scalable and maintainable applications. |
Event Sourcing and CQRS(Command and Query Responsibilty Segregation)
The factors represent a set of guidelines or best practices for portable, resilient applications that will thrive in cloud environments(specifically software as a service applications)
- There should be a one-to-one association between a versioned codebase (for example,an IT repository) and a deployed service. The same codebase is used for many deployments.
- Services should explicitly declare all dependencies, and should not rely on the presence of system-level tools or libraries.
- Configuration that varies between deployment environments should be stored in the environment (specifically in environment variables).
- All backing services are treated as attached resources, which are managed (attached and detached) by the execution environment.
- The delivery pipeline should have strictly separate stages: Build, release, and run.
- Applications should be deployed as one or more stateless processes. Specifically, transient processes must be stateless and share nothing. Persisted data should be stored in an appropriate backing service.
- Self-contained services should make themselves available to other services by listening on a specified port.
- Concurrency is achieved by scaling individual processes (horizontal scaling).
- Processes must be disposable: Fast startup and graceful shutdown behaviors lead to a more robust and resilient system.
- All environments, from local development to production, should be as similar as possible.
- Applications should produce logs as event streams (for example, writing to stdout and stderr), and trust the execution environment to aggregate streams.
- If admin tasks are needed, they should be kept in source control and packaged alongside the application to ensure that it is run with the same environment as the application.
| Load/Stress Test |
|---|
| ab - Apache HTTP server benchmarking tool |
| Apache JMeter |
| Siege |
| FunkLoad |
| loader.io |
| Gatling |
| wrk, Modern HTTP benchmarking tool |
- CAP 定理（CAP theorem, Consistency, Availability, Partition tolerance）
- Distributed Systems for fun and profit
- 8 fallacies of distributed computing
- FLP 不可能性的
- Raft A Consensus Algorithm visualization of Raft
Soft
| Oracle | Postgresql | Mysql | SQLite | |
|---|---|---|---|---|
| Introduction | A very powerful, embedded relational database management system | |||
| Relational Database | ✔️ | ✔️ | ✔️ | ✔️ |
| Advantage | An open-source SQL standard compliant RDBMS,Strong community,Strong third-party support,Extensible,Objective | Easy to work,Feature rich,Secure, Scalabel, Speedy | File based, Standards-aware | |
| Disadvantage | Performance,Hosting | Function limitations, Reliability issues, Stagnated development | No user management, Lack of possibility to tinker with for additional performance | |
| When to use | Data Integrity, Complex custom procedures,Integration,complex design | Distributed operations, Hight security,Web site or Web app, Custom solution | Embedded app, Disk access replacement, Testing | |
| When not to use | Speed,Simple set ups,Replication | SQL Compliance, Concurrency, Full-text search | Multi-User App, High write volumes |
| postgresql |
|---|
| postgresql |
| pgAdmin GUI for Postgresql |
| [postgresql index]](https://github.com/digoal/blog/blob/master/201706/20170627_01.md) |
| MySQL |
|---|
| MySQL8.0 new feature 支持文档存储 |
Greenplum an open-source massively parallel data platform for analytics, machine learning and AI
MyBatis is a first class persistence framework with support for custom SQL, stored procedures and advanced mappings. MyBatis Web Example
System Properties Comparison Memcached vs. Redis
Redis is better than Memcache
缓存设计的要点 1.缓存穿透，避免缓存穿透 2.缓存雪崩，加锁或者后台更新 3.缓存热点，备份 多副本
- Abstract Factory 抽象工厂, 提供一个接口，可以创建一系列相关或相互依赖的对象，而无须指定它们具体的类
- Builder 构建器，将一个复杂类的构建与其表示相分离，使得相同的构件过程能够得出不同的表示
- Factory Method 工厂方法，定义一个建对象的接口，但由子类决定需要实例化哪一个类，工厂方法使得子类实例化的过程推迟
- Prototype 原型，用原型实例指定创建对象的类型，并且通过复制这个原型来创建新的对象
- Singleton 单例，保证一个类只有一个实例，并提供一个访问它的全局点
- Adapter 适配器，将一个类的接口转换成用户希望的另一个接口，使原本不相兼容的接口得以协同工作
- Bridge 桥接，将类的抽象部分和它的实现部分分离开来，使得它们可以独立地变化
- Composite 组合，将对象组合成树型结构以表示“整体-部分”的层次结构，使得用户对单个对象和组合对象的使用具有一致性
- Decorator 装饰器，动态地给一个对象添加一些额外的功能，它提供了子类扩展功能的一个灵活的替代，比派生一个子类更加灵活
- Facade 外观模式，定义一个高层次接口，为子系统中的一组接口提供一个一致的外观，从而简化了该子系统的使用
- Flyweight 享元模式，提供支持大量细粒度对象共享的有效方法
- Proxy 代理，为其他对象提供一种代理以控制这个对象的访问
- Chain of Responsiblity 责任链，通过给多个对象处理请求的机会，减少请求的发送者与接收者之间的耦合。将接收对象链接起来，在链中传递请求，直到有一个对象处理这个请求
- Command 命令，将一个请求封装为一个对象，从而可用不同的请求对客户进行参数化，将请求排队或记录请求日志，支持可撤销操作
- Iterpreter 解释器，给定一种语言，定义它的文法表示，并定义一个解释器，该解释器用来根据文法表示来解释语言中的句子
- Iterator 迭代器，提供一种方法来顺序访问一个聚合对象中的各个元素，而不需要暴露该对象的内部表示
- Mediator 中介者，用一个中介对象来封装一系列的对象交互。它使各对象不需要显示地相互调用，从而达到低耦合，还可以独立地改变对象建的交互
- Memento 备忘录，在不破坏封装性的前提下，捕获一个对象的内部状态，并在该对象之外保存这个状态，从而可以在以后将该对象恢复到原先保存的状态
- Observer 观察者，定义对象间的一种一对多的依赖关系，当一个对象的状态发生变化时，所有依赖于它的对象都得到通知并自动更新
- State 状态，允许一个对象在其内部状态改变时改变它的行为
- Strategy 策略，定义一系列算法，把它们一个个封装起来，并且使它们之间可互相替换，从而让算法可以独立于使用它的用户而变化
- Template Method 模板方法，定义一个操作中的算法骨架，而将一些步骤延迟到子类中，使得子类可以不改变一个算法的结构即可重新定义算法的某些特定步骤
- Visitor 访问者，表示一个作用于某对象结构中的各元素的操作，使得在不改变各元素的类的前提下定义作用于这些元素的新操作
又称失效模式与后果分析、失效模式与效应分析、故障模式与后果分析等。
- 找出产品或过程中潜在的故障模式
- 根据相应的评价体系对找出的潜在故障模式进行风险量化评估
- 列出故障的起因/机理，寻找预防或改进措施
ElasticSearch https://www.elastic.co/guide/en/elasticsearch/reference/current/_basic_concepts.html
Apache Solr
Sphinx
Apache lucene
https://mp.weixin.qq.com/s/FUHnDFiHWeJC1J6-jAh9mA
Based On HBase
Small File Server
淘宝 TFS、京东 JFS、Facebook 的 Haystack
| Client | Written in | License | Access API |
|---|---|---|---|
| Ceph | C++ | LGPL | librados (C, C++, Python, Ruby), S3, Swift, FUSE |
| BeeGFS | C / C++ | FRAUNHOFER FS (FhGFS) EULA,[1] GPLv2 client | POSIX |
| GlusterFS | C | GPLv3 | libglusterfs, FUSE, NFS, SMB, Swift, libgfapi |
| MooseFS | C | GPLv2 | POSIX, FUSE |
| Quantcast File System | C | Apache License 2.0 | C++ client, FUSE (C++ server: MetaServer and ChunkServer are both in C++) |
| Spectrum Scale (GPFS) | C, C++ | Proprietary | POSIX, NFS, SMB, Swift |
| Lustre | C | GPLv2 | POSIX, liblustre, FUSE |
| MapR-FS | C, C++ | Proprietary | POSIX, NFS, FUSE, S3 |
| OpenAFS | C | IBM Public License | Virtual file system, Installable File System |
| scality | C | Proprietary | fuse, NFS file system, rest, AWS S3 |
| Tahoe-LAFS | Python | GNU GPL 2+ and other[5] | HTTP (browser or CLI), SFTP, FTP, FUSE via SSHFS, pyfilesystem |
| HDFS | Java | Apache License 2.0 | Java and C client, HTTP |
| XtreemFS | Java, C++ | BSD License | libxtreemfs (Java, C++), FUSE |
| Ori[6] | C, C++ | MIT | libori, FUSE |
HDFS vs Ceph vs GlusterFS
http://iopscience.iop.org/article/10.1088/1742-6596/513/4/042014/pdf
MooseFS 适合大中文件
Wireshark is the world’s foremost and widely-used network protocol analyzer.
RawCap is a free command line network sniffer for Windows that uses raw sockets.
C10K Problem C10K Problem翻译 http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html
Nginx is an open source web server and reverse proxy Load_balancing (http://nginx.org/en/docs/http/load_balancing.html)
kong基于 OpenResty 的 API 网关服务和网关服务管理层.
Prometheus Power your metrics and alerting with a leading open-source monitoring solution Learning Prometheus
Tyk Open Source API Gateway written in Go
https://shardingsphere.apache.org/document/current/en/overview/
SLF4J
ELK(Elasticsearch Logstash Kibana) a Centralized Logging stack.
fluentd Open-Source Log Collector
a better conceptual model is to treat logs as time-ordered streams: there is no beginning or end, but rather an ongoing, collated collection of events which we may wish to view in realtime as they happen (e.g. via tail -f or heroku logs --tail) or which we may wish to search in some time window (e.g. via grep or Splunk). fluentd Docker
MQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol
Elastic-Job is a distributed scheduled job framework, based on Quartz and Zookeeper.
A lightweight distributed task scheduling framework.（分布式任务调度平台XXL-JOB）
Netflix Conductor is a microservices orchestration engine
Zeebe is a workflow engine for microservices orchestration.
BPMN, Business Process Model and Notation Flowable (deprecated) Activiti, after 6.0 it was not development
ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files Grammars written for ANTLR v4; expectation that the grammars are free of actions.