kcp

package module
v1.0.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 24, 2019 License: Apache-2.0 Imports: 28 Imported by: 0

README

Official site: http://rpcx.site

License GoDoc travis Go Report Card coveralls QQ群

Cross-Languages

you can use other programming languages besides Go to access rpcx services.

  • rpcx-gateway: You can write clients in any programming languages to call rpcx services via rpcx-gateway
  • http invoke: you can use the same http requests to access rpcx gateway
  • Java Services/Clients: You can use rpcx-java to implement/access rpcx servies via raw protocol.

If you can write Go methods, you can also write rpc services. It is so easy to write rpc applications with rpcx.

Installation

install the basic features:

go get -u -v git.parallelcoin.io/dev/rpcx/...

If you want to use reuseportquickcp, zookeeper, etcd, consul registry, use those tags to go getgo build or go run. For example, if you want to use all features, you can:

go get -u -v -tags "reuseport quic kcp zookeeper etcd consul ping rudp utp" git.parallelcoin.io/dev/rpcx/...

tags:

  • quic: support quic transport
  • kcp: support kcp transport
  • zookeeper: support zookeeper register
  • etcd: support etcd register
  • consul: support consul register
  • ping: support network quality load balancing
  • reuseport: support reuseport

Features

rpcx is a RPC framework like Alibaba Dubbo and Weibo Motan.

rpcx 3.0 has been refactored for targets:

  1. Simple: easy to learn, easy to develop, easy to intergate and easy to deploy
  2. Performance: high perforamnce (>= grpc-go)
  3. Cross-platform: support raw slice of bytes, JSON, Protobuf and MessagePack. Theoretically it can be used with java, php, python, c/c++, node.js, c# and other platforms
  4. Service discovery and service governance: support zookeeper, etcd and consul.

It contains below features

  • Support raw Go functions. There's no need to define proto files.
  • Pluggable. Features can be extended such as service discovery, tracing.
  • Support TCP, HTTP, QUIC and KCP
  • Support multiple codecs such as JSON, Protobuf, MessagePack and raw bytes.
  • Service discovery. Support peer2peer, configured peers, zookeeper, etcd, consul and mDNS.
  • Fault tolerance:Failover, Failfast, Failtry.
  • Load banlancing:support Random, RoundRobin, Consistent hashing, Weighted, network quality and Geography.
  • Support Compression.
  • Support passing metadata.
  • Support Authorization.
  • Support heartbeat and one-way request.
  • Other features: metrics, log, timeout, alias, circuit breaker.
  • Support bidirectional communication.
  • Support access via HTTP so you can write clients in any programming languages.
  • Support API gateway.
  • Support backup request, forking and broadcast.

rpcx uses a binary protocol and platform-independent, which means you can develop services in other languages such as Java, python, nodejs, and you can use other prorgramming languages to invoke services developed in Go.

There is a UI manager: rpcx-ui.

Performance

Test results show rpcx has better performance than other rpc framework except standard rpc lib.

The benchmark code is at rpcx-benchmark.

Listen to others, but test by yourself.

Test Environment

  • CPU: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz, 32 cores
  • Memory: 32G
  • Go: 1.9.0
  • OS: CentOS 7 / 3.10.0-229.el7.x86_64

Use

  • protobuf
  • the client and the server on the same server
  • 581 bytes payload
  • 500/2000/5000 concurrent clients
  • mock processing time: 0ms, 10ms and 30ms

Test Result

mock 0ms process time

ThroughputsMean LatencyP99 Latency

mock 10ms process time

ThroughputsMean LatencyP99 Latency

mock 30ms process time

ThroughputsMean LatencyP99 Latency

Examples

You can find all examples at rpcx-ecosystem/rpcx-examples3.

The below is a simple example.

Server

    // define example.Arith
    ……

    s := server.NewServer()
	s.RegisterName("Arith", new(example.Arith), "")
	s.Serve("tcp", addr)

Client

    // prepare requests
    ……

    d := client.NewPeer2PeerDiscovery("tcp@"+addr, "")
	xclient := client.NewXClient("Arith", client.Failtry, client.RandomSelect, d, client.DefaultOption)
	defer xclient.Close()
	err := xclient.Call(context.Background(), "Mul", args, reply, nil)

Contribute

see contributors.

Welcome to contribute:

  • submit issues or requirements
  • send PRs
  • write projects to use rpcx
  • write tutorials or articles to introduce rpcx

License

Apache License, Version 2.0

Documentation

Overview

Package kcp - A Fast and Reliable ARQ Protocol

Index

Constants

View Source
const (
	IKCP_RTO_NDL     = 30  // no delay min rto
	IKCP_RTO_MIN     = 100 // normal min rto
	IKCP_RTO_DEF     = 200
	IKCP_RTO_MAX     = 60000
	IKCP_CMD_PUSH    = 81 // cmd: push data
	IKCP_CMD_ACK     = 82 // cmd: ack
	IKCP_CMD_WASK    = 83 // cmd: window probe (ask)
	IKCP_CMD_WINS    = 84 // cmd: window size (tell)
	IKCP_ASK_SEND    = 1  // need to send IKCP_CMD_WASK
	IKCP_ASK_TELL    = 2  // need to send IKCP_CMD_WINS
	IKCP_WND_SND     = 32
	IKCP_WND_RCV     = 32
	IKCP_MTU_DEF     = 1400
	IKCP_ACK_FAST    = 3
	IKCP_INTERVAL    = 100
	IKCP_OVERHEAD    = 24
	IKCP_DEADLINK    = 20
	IKCP_THRESH_INIT = 2
	IKCP_THRESH_MIN  = 2
	IKCP_PROBE_INIT  = 7000   // 7 secs to probe window size
	IKCP_PROBE_LIMIT = 120000 // up to 120 secs to probe window
)

Variables

This section is empty.

Functions

func Dial

func Dial(raddr string) (net.Conn, error)

Dial connects to the remote address "raddr" on the network "udp"

func Listen

func Listen(laddr string) (net.Listener, error)

Listen listens for incoming KCP packets addressed to the local address laddr on the network "udp",

Types

type BlockCrypt

type BlockCrypt interface {
	// Encrypt encrypts the whole block in src into dst.
	// Dst and src may point at the same memory.
	Encrypt(dst, src []byte)

	// Decrypt decrypts the whole block in src into dst.
	// Dst and src may point at the same memory.
	Decrypt(dst, src []byte)
}

BlockCrypt defines encryption/decryption methods for a given byte slice. Notes on implementing: the data to be encrypted contains a builtin nonce at the first 16 bytes

func NewNoneBlockCrypt

func NewNoneBlockCrypt(key []byte) (BlockCrypt, error)

NewNoneBlockCrypt does nothing but copying

func NewSalsa20BlockCrypt

func NewSalsa20BlockCrypt(key []byte) (BlockCrypt, error)

NewSalsa20BlockCrypt https://en.wikipedia.org/wiki/Salsa20

func NewSimpleXORBlockCrypt

func NewSimpleXORBlockCrypt(key []byte) (BlockCrypt, error)

NewSimpleXORBlockCrypt simple xor with key expanding

func NewTripleDESBlockCrypt

func NewTripleDESBlockCrypt(key []byte) (BlockCrypt, error)

NewTripleDESBlockCrypt https://en.wikipedia.org/wiki/Triple_DES

func NewTwofishBlockCrypt

func NewTwofishBlockCrypt(key []byte) (BlockCrypt, error)

NewTwofishBlockCrypt https://en.wikipedia.org/wiki/Twofish

func NewXTEABlockCrypt

func NewXTEABlockCrypt(key []byte) (BlockCrypt, error)

NewXTEABlockCrypt https://en.wikipedia.org/wiki/XTEA

type Entropy

type Entropy interface {
	Init()
	Fill(nonce []byte)
}

Entropy defines a entropy source

type KCP

type KCP struct {
	// contains filtered or unexported fields
}

KCP defines a single KCP connection

func NewKCP

func NewKCP(conv uint32, output output_callback) *KCP

NewKCP create a new kcp control object, 'conv' must equal in two endpoint from the same connection.

func (*KCP) Check

func (kcp *KCP) Check() uint32

Check determines when should you invoke ikcp_update: returns when you should invoke ikcp_update in millisec, if there is no ikcp_input/_send calling. you can call ikcp_update in that time, instead of call update repeatly. Important to reduce unnacessary ikcp_update invoking. use it to schedule ikcp_update (eg. implementing an epoll-like mechanism, or optimize ikcp_update when handling massive kcp connections)

func (*KCP) Input

func (kcp *KCP) Input(data []byte, regular, ackNoDelay bool) int

Input when you received a low level packet (eg. UDP packet), call it regular indicates a regular packet has received(not from FEC)

func (*KCP) NoDelay

func (kcp *KCP) NoDelay(nodelay, interval, resend, nc int) int

NoDelay options fastest: ikcp_nodelay(kcp, 1, 20, 2, 1) nodelay: 0:disable(default), 1:enable interval: internal update timer interval in millisec, default is 100ms resend: 0:disable fast resend(default), 1:enable fast resend nc: 0:normal congestion control(default), 1:disable congestion control

func (*KCP) PeekSize

func (kcp *KCP) PeekSize() (length int)

PeekSize checks the size of next message in the recv queue

func (*KCP) Recv

func (kcp *KCP) Recv(buffer []byte) (n int)

Recv is user/upper level recv: returns size, returns below zero for EAGAIN

func (*KCP) ReserveBytes

func (kcp *KCP) ReserveBytes(n int) bool

ReserveBytes keeps n bytes untouched from the beginning of the buffer the output_callback function should be aware of this return false if n >= mss

func (*KCP) Send

func (kcp *KCP) Send(buffer []byte) int

Send is user/upper level send, returns below zero for error

func (*KCP) SetMtu

func (kcp *KCP) SetMtu(mtu int) int

SetMtu changes MTU size, default is 1400

func (*KCP) Update

func (kcp *KCP) Update()

Update updates state (call it repeatedly, every 10ms-100ms), or you can ask ikcp_check when to call it again (without ikcp_input/_send calling). 'current' - current timestamp in millisec.

func (*KCP) WaitSnd

func (kcp *KCP) WaitSnd() int

WaitSnd gets how many packet is waiting to be sent

func (*KCP) WndSize

func (kcp *KCP) WndSize(sndwnd, rcvwnd int) int

WndSize sets maximum window size: sndwnd=32, rcvwnd=32 by default

type Listener

type Listener struct {
	// contains filtered or unexported fields
}

Listener defines a server which will be waiting to accept incoming connections

func ListenWithOptions

func ListenWithOptions(laddr string, block BlockCrypt, dataShards, parityShards int) (*Listener, error)

ListenWithOptions listens for incoming KCP packets addressed to the local address laddr on the network "udp" with packet encryption, dataShards, parityShards defines Reed-Solomon Erasure Coding parameters

func ServeConn

func ServeConn(block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*Listener, error)

ServeConn serves KCP protocol for a single packet connection.

func (*Listener) Accept

func (l *Listener) Accept() (net.Conn, error)

Accept implements the Accept method in the Listener interface; it waits for the next call and returns a generic Conn.

func (*Listener) AcceptKCP

func (l *Listener) AcceptKCP() (*UDPSession, error)

AcceptKCP accepts a KCP connection

func (*Listener) Addr

func (l *Listener) Addr() net.Addr

Addr returns the listener's network address, The Addr returned is shared by all invocations of Addr, so do not modify it.

func (*Listener) Close

func (l *Listener) Close() error

Close stops listening on the UDP address. Already Accepted connections are not closed.

func (*Listener) SetDSCP

func (l *Listener) SetDSCP(dscp int) error

SetDSCP sets the 6bit DSCP field of IP header

func (*Listener) SetDeadline

func (l *Listener) SetDeadline(t time.Time) error

SetDeadline sets the deadline associated with the listener. A zero time value disables the deadline.

func (*Listener) SetReadBuffer

func (l *Listener) SetReadBuffer(bytes int) error

SetReadBuffer sets the socket read buffer for the Listener

func (*Listener) SetReadDeadline

func (l *Listener) SetReadDeadline(t time.Time) error

SetReadDeadline implements the Conn SetReadDeadline method.

func (*Listener) SetWriteBuffer

func (l *Listener) SetWriteBuffer(bytes int) error

SetWriteBuffer sets the socket write buffer for the Listener

func (*Listener) SetWriteDeadline

func (l *Listener) SetWriteDeadline(t time.Time) error

SetWriteDeadline implements the Conn SetWriteDeadline method.

type Snmp

type Snmp struct {
	BytesSent        uint64 // bytes sent from upper level
	BytesReceived    uint64 // bytes received to upper level
	MaxConn          uint64 // max number of connections ever reached
	ActiveOpens      uint64 // accumulated active open connections
	PassiveOpens     uint64 // accumulated passive open connections
	CurrEstab        uint64 // current number of established connections
	InErrs           uint64 // UDP read errors reported from net.PacketConn
	InCsumErrors     uint64 // checksum errors from CRC32
	KCPInErrors      uint64 // packet iput errors reported from KCP
	InPkts           uint64 // incoming packets count
	OutPkts          uint64 // outgoing packets count
	InSegs           uint64 // incoming KCP segments
	OutSegs          uint64 // outgoing KCP segments
	InBytes          uint64 // UDP bytes received
	OutBytes         uint64 // UDP bytes sent
	RetransSegs      uint64 // accmulated retransmited segments
	FastRetransSegs  uint64 // accmulated fast retransmitted segments
	EarlyRetransSegs uint64 // accmulated early retransmitted segments
	LostSegs         uint64 // number of segs infered as lost
	RepeatSegs       uint64 // number of segs duplicated
	FECRecovered     uint64 // correct packets recovered from FEC
	FECErrs          uint64 // incorrect packets recovered from FEC
	FECParityShards  uint64 // FEC segments received
	FECShortShards   uint64 // number of data shards that's not enough for recovery
}

Snmp defines network statistics indicator

var DefaultSnmp *Snmp

DefaultSnmp is the global KCP connection statistics collector

func (*Snmp) Copy

func (s *Snmp) Copy() *Snmp

Copy make a copy of current snmp snapshot

func (*Snmp) Header

func (s *Snmp) Header() []string

Header returns all field names

func (*Snmp) Reset

func (s *Snmp) Reset()

Reset values to zero

func (*Snmp) ToSlice

func (s *Snmp) ToSlice() []string

ToSlice returns current snmp info as slice

type UDPSession

type UDPSession struct {
	// contains filtered or unexported fields
}

UDPSession defines a KCP session implemented by UDP

func DialWithOptions

func DialWithOptions(raddr string, block BlockCrypt, dataShards, parityShards int) (*UDPSession, error)

DialWithOptions connects to the remote address "raddr" on the network "udp" with packet encryption

func NewConn

func NewConn(raddr string, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error)

NewConn establishes a session and talks KCP protocol over a packet connection.

func (*UDPSession) Close

func (s *UDPSession) Close() error

Close closes the connection.

func (*UDPSession) GetConv

func (s *UDPSession) GetConv() uint32

GetConv gets conversation id of a session

func (*UDPSession) LocalAddr

func (s *UDPSession) LocalAddr() net.Addr

LocalAddr returns the local network address. The Addr returned is shared by all invocations of LocalAddr, so do not modify it.

func (*UDPSession) Read

func (s *UDPSession) Read(b []byte) (n int, err error)

Read implements net.Conn

func (*UDPSession) RemoteAddr

func (s *UDPSession) RemoteAddr() net.Addr

RemoteAddr returns the remote network address. The Addr returned is shared by all invocations of RemoteAddr, so do not modify it.

func (*UDPSession) SetACKNoDelay

func (s *UDPSession) SetACKNoDelay(nodelay bool)

SetACKNoDelay changes ack flush option, set true to flush ack immediately,

func (*UDPSession) SetDSCP

func (s *UDPSession) SetDSCP(dscp int) error

SetDSCP sets the 6bit DSCP field of IP header, no effect if it's accepted from Listener

func (*UDPSession) SetDUP

func (s *UDPSession) SetDUP(dup int)

SetDUP duplicates udp packets for kcp output, for testing purpose only

func (*UDPSession) SetDeadline

func (s *UDPSession) SetDeadline(t time.Time) error

SetDeadline sets the deadline associated with the listener. A zero time value disables the deadline.

func (*UDPSession) SetMtu

func (s *UDPSession) SetMtu(mtu int) bool

SetMtu sets the maximum transmission unit(not including UDP header)

func (*UDPSession) SetNoDelay

func (s *UDPSession) SetNoDelay(nodelay, interval, resend, nc int)

SetNoDelay calls nodelay() of kcp https://github.com/skywind3000/kcp/blob/master/README.en.md#protocol-configuration

func (*UDPSession) SetReadBuffer

func (s *UDPSession) SetReadBuffer(bytes int) error

SetReadBuffer sets the socket read buffer, no effect if it's accepted from Listener

func (*UDPSession) SetReadDeadline

func (s *UDPSession) SetReadDeadline(t time.Time) error

SetReadDeadline implements the Conn SetReadDeadline method.

func (*UDPSession) SetStreamMode

func (s *UDPSession) SetStreamMode(enable bool)

SetStreamMode toggles the stream mode on/off

func (*UDPSession) SetWindowSize

func (s *UDPSession) SetWindowSize(sndwnd, rcvwnd int)

SetWindowSize set maximum window size

func (*UDPSession) SetWriteBuffer

func (s *UDPSession) SetWriteBuffer(bytes int) error

SetWriteBuffer sets the socket write buffer, no effect if it's accepted from Listener

func (*UDPSession) SetWriteDeadline

func (s *UDPSession) SetWriteDeadline(t time.Time) error

SetWriteDeadline implements the Conn SetWriteDeadline method.

func (*UDPSession) SetWriteDelay

func (s *UDPSession) SetWriteDelay(delay bool)

SetWriteDelay delays write for bulk transfer until the next update interval

func (*UDPSession) Write

func (s *UDPSession) Write(b []byte) (n int, err error)

Write implements net.Conn

func (*UDPSession) WriteBuffers

func (s *UDPSession) WriteBuffers(v [][]byte) (n int, err error)

WriteBuffers write a vector of byte slices to the underlying connection

Directories

Path Synopsis
Package client is a generated protocol buffer package.
Package client is a generated protocol buffer package.
tool
xgen/parser
Package parser parses Go code and keeps track of all the types defined and provides access to all the constants defined for an int type.
Package parser parses Go code and keeps track of all the types defined and provides access to all the constants defined for an int type.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL