Академический Документы
Профессиональный Документы
Культура Документы
www.browan.com
BROWAN COMMUNICATIONS
INCORPORTATION
April 22, 2016
0
Public
Table of Contents
Table of Contents..................................................................................................1
Chapter 1: Introduction...............................................................................................3
1.1 Purpose........................................................................................................3
1.2 Scope...........................................................................................................3
1.3 Overview......................................................................................................3
1.4 Definitions and Acronyms..............................................................................4
Chapter 2: Design Considerations................................................................................5
Chapter 3: Architecture................................................................................................8
3.1 Overview......................................................................................................8
3.2 Modules.....................................................................................................10
3.2.1 Basic Function Modules.....................................................................10
3.2.2 Audio and Video Module....................................................................11
Public
4.4.11 Video Conference Subscription...........................................................36
4.4.12 One-way Video Control Mode............................................................38
Public
Introduction
1.1 Purpose
It mainly introduces the architecture and basic design considerations of FreePP Mobile
SDK, and describes functions, call method and workflow of SDK API interface in details.
1.2 Scope
It is suitable for FreePP Mobile SDK developers and software testing engineer to read
and discuss.
1.3 Overview
FreePP Mobile SDK (iOS version) is developed with Objective-C language and iOS
SDK 7.1 class library under XCode 5.1 version, and is compatible with iOS 5.0/6.0/7.0
system and higher.
Through FreePP Mobile SDK, third-party application programmer (APP for short) can
realize authentication, communication number allocation and query, instant message, A/V
(audio and video) communication, and other functions, including:
Binding and query of APP account with the unified communication number inside
instant communication system (communication no. for short)
One-to-one instant message, including text or file message
Instant group message
Group member control, including: invite, mute, join and leave
One-to-one audio/video call
Audio/video conference (including: client or server audio mixing)
Real-time push to talk
Besides, for different operating systems and CPU processors, SDK adopts hardware
acceleration optimization algorithm in modules with high computational complexity, like
A/V compression codec, and compiles the algorithm for armv7 and arm64 processors
respectively to ensure smooth user experience.
The content of this document mainly includes: SDK architecture, SDK internal function
Public
modules, SDK design considerations, SDK package, API interface design and workflow.
Notification Server
Configuration Server
Foreign Exchange Subscriber
Foreign Exchange Office
Apple Push Notification Service
FreePP Push Notification Service
Message Exchange Center
Public
Design Considerations
To ensure the interoperability, compatibility and openness of SDK, support third-party
accounts in various forms (e.g. nickname, Email address, phone no. or APP account, etc.),
hide the implementation details of underlying communication system, and reduce APP
development complexity, SDK follows the basic principles below in design:
Unified User Identifier
Within the system, SDK identifies each user uniquely with its communication no., APP
can associate a specific APP account with the allocated communication no. by binding
method. In this way, for different forms of APP accounts, a unified user identification
management and control mechanism inside communication system can be realized by
communication no., to ensure interconnection and interoperability among different APPs.
Besides, when APP calls SDK API interface, communication no. should be used as target
parameter, but for end-user, it is transparent and invisible, APP can get corresponding
communication no. of each APP account, through SDK query account interface.
The relation between APP account and communication no. is shown below,
Note: SDK can support intercommunication between different forms of APP accounts
through unified user identification, i.e., communication no.
Unified Message Format
In SDK system, MIME coding format is unified to define various types of instant
message, the basic format is type/subtype, e.g.:
Message type
MIME code
Text string
text/plain
Public
Image file
image/{png,jpeg...}
Audio file
audio/{amr,3gp,mp3}
Video file
video/{mpeg ,mp4,avi,asf,mov}
application/{pdf ,msword,octet-stream...}
JSON string
application/json
Note: types of instant message are not limited to the definition mentioned above, APP
can expand new MIME type and corresponding function to suit its own needs, while SDK
can ensure transparent transmission of MIME type when transmitting message.
Unified Character Coding Format
To be compatible with strings in different languages and avoid messy code between
different languages or platforms, SDK only adopts UTF-8 code inside the system.
Both input parameters and return values of SDK API interface adopt UTF-8 coding
format, SDK developer needs to complete conversion of related string coding formats in
upper layer. Meanwhile, SDK also provides auxiliary interface function for interconversion
between UNICODE string and UTF-8 string.
Asynchronous Operations of Interface
To reduce the operation coupling between APP and SDK, and improve program
efficiency and responsiveness, all interfaces involving message and conversation functions
(usually need to assess network and take a long time) handle APPs call requests with nonblocking mode, i.e. APP can return and continue to run immediately after calling SDK API
interface, while SDK keeps processing in background without blocking the APP, and return
results to APP asynchronously by corresponding callback function, therefore, APP doesnt
need to wait, otherwise may cause main thread blocking, making App become unresponsive.
Besides, message/call request ID is adopted between APP and SDK to match
asynchronous request and response. E.g.:
When APP sends message, SDK interface will return the message ID to APP
immediately, meanwhile transmitting message in background with multithread technology;
After sending message successfully, SDK will notify APP by callback function, and
Public
Public
Architecture
3.1 Overview
FreePP Mobile SDK has two main functions: user identity authentication/instant
message and A/V call, of which the former one is SDKs basic function, as well as the
infrastructure for client to access the instant communication system.
Basic architecture of user identity authentication/instant message function is shown
below,
In the above figure, there are following layers in SDK, by call relations between function
blocks:
Interface layer: mask the implementation details inside middleware, and provide
easy-to-use services of user identity authentication and instant message for App.
Service layer: provide user identity management and control, and workflow of
various instant communication services.
Communication layer: provide interactive operation between client and server, and
ensure that client can be notified in time.
Public
Based on the above functions, A/V communication also provides one-to-one video call
and multi-party A/V conference functions.
Basic architecture of SDKs A/V function is shown below,
In the above figure, there are following layers inside SDK, by call relations between
function blocks:
Interface layer: hide the implementation details inside middleware, and provide
easy-to-use A/V communication services for APP.
Call control and information processing layer: provide workflow of call setup,
maintenance, and release, and A/V stream processing and network adaptive
function.
Moreover, call state machine is the core of A/V call module, mainly responsible for
recording and managing the current state of each call, and makes corresponding processing
action according to user operation or request. SDK call state machine implements call state
control according to following state transition diagram (Figure 3)
FreePP Mobile SDK on iOS API Document- 9 -
Public
In the above figure, IDLE refers to call standby, i.e. SDK returns to this state before
initiating or finishing a call; INITIATE refers to call initiating state, i.e. both parties of a
call are in this state after initiating call and before answering; HANGUP refers to call
finishing state, i.e. both parties of a call are in this state if either party hangs up first. Besides,
once in HANGUP state, SDK will transfer to IDLE state automatically.
When using call modules, APP needs to display corresponding user interface
according to current call state or SDK call state notification, and perform
corresponding call control operations.
3.2 Modules
3.2.1 Basic Function Modules
Basic function modules include user identity authentication/instant message function.
Layer
Module
Service layer
Description
Public
Communication
layer
Protocol layer
message
File transfer
Local storage
Group
management
Long
connection
Heartbeat service
Receiving of push notification
Web service
Web Service
Module
Description
Video engine
Audio engine
Public
HTTP
IAX2/UDP
Relations between above SDK libraries and API interfaces are shown in following table
(characters in blue are for Android, and that in black are for iOS and cross platform):
SDK
version
Basic
version
Reference relation
Dependency relation
Include/Import (.a/.so)
FreePPSDK.h/jar
libFreePPSDKBasic.a/so
Function
1)Basic
functions
(initialization and user
management);
2)Instant message;
Call
FreePPSDK.h/jar
libFreePPSDKStd.a/so
1)Basic
functions
version
FreePPSDKStream. or
(initialization and user
h/jar
libFreePPSDKStdx.a/so
management);
2)Instant message;
3)One-to-one
video
call;
4)Multi-party
A/V
conference;
Enhanced FreePPSDK.h/jar
libFreePPSDKPro.a/so
1)Basic
functions
version
FreePPSDKStream. or
(initialization and user
h/jar
libFreePPSDKProx.a/so
management);
FreePPSDKPTT.h/ja
2)Instant message;
r
3)One-to-one
video
call;
4)A/V conference;
5)Audio intercom;
Note: call version and enhanced version can generate two library files, respectively
Public
corresponding to two solutions--client audio mixing and server audio mixing (the
one with x behind is server audio mixing). When using SDK, developer can
choose either of the two library files as required by service.
When the third-party APP uses SDK, the following Framework and Library need to be
linked to APP project:
If only use basic version SDK, add framework and Library as shown below to
project,
If use call version SDK, be sure to add Library (as shown below) to project,
Call version SDK needs specific compilation options, which are shown below,
Public
When third party references call SDK library files, be sure to add macro definition
FREEPP_SDK_STREAM_API to use API method in SDK, as shown below,
Public
When the third-party references call version SDK library file for conference
function, be sure to add macro definition FREEPP_SDK_STREAM_CONF_API
to use the relevant conference API method in SDK, as shown below:
Public
SDK API
FreePP Mobile SDK mainly has two types of API interfaces: message API and call API.
In general, API interfaces in SDK are all synchronous (blocking) interfaces; asynchronous
(non-blocking) interfaces will be specified in special description.
Moreover, any interface marked with CALLBACK is callback interface, and APP needs
to realize such interface function in delegate class, meanwhile avoid performing the long
running tasks, to prevent blocking the worker thread of SDK.
Parameters
Return values
Notice
+ (BOOL)initialize
appKey
Boolean-client
initialization result, if
client module.
delegate
Any pointer
type,
third-party
APPs delegate.
+(NSString
appUserAccount
String-FreePP
ID
*)bindAppAccount
account
of accounts).
If
mustAuth
NULL,
illegitimate,
should
binding
perform
connect
remote
to
server
to
identification,
return
value
is
FreePP system.
AppKey
is
If
or
For
fails.
callerName
and
carried
code table.
outbound call.
out
and
in
Public
String, users local nickname,
mainly used to set callers name
in outbound call.
callerNumber
String, users phone no., mainly
used to set callers phone no. in
outbound call, standard E.164
format.
+
(NSDictionary
appAccountArray
String dictionary-list of
*)queryFreePPIDB
yAppAccount
target user.
ID
corresponding
account
(Hash
key=value).
If
and
App
account.
table:
error
accounts
(NSDictionary
freePPIDArray
String dictionary-list of
*)queryAppAccou
ntByFreePPID
target user.
ID
corresponding
account
(Hash
and
App
account.
table:
key=value).
If
occurs
peer-party
or
error
NULL;
if
accounts
name
Set
SDKs
(int)setParameter
working parameters.
value
Note:
table.
supported
for
please
internal
currently
parameters,
refer
to
Parameter
Configuration Table.
+
(NSString
*)getParameter
name
String-parameter
Get
SDKs
value. If it doesnt
parameters.
internal
No
Integer, if no error
(int)getLastErrorC
Last
ode
invocation.
failed
interface
Public
section 4.4 error code
table)
No
Integer, if no error
(int)logoutAppAcc
occurs, return 0.
account.
ount
Error
code:
FreePP
-15:
SDK,
uninitialized.
delete
local
login
by
No
(void)onSystemEve
including:
corresponding to current
nt
account.
out
Operation
Realization
Notice
requirement
Version
get
No
Rootcs
get/set
Yes
max_group_member
get
No
video_orientation
get/set
No
Video
display
orientation:
get/set
No
Parameter
Return value
Notice
+ (NSString *)sendMessage
dstFreePPID
String-
message
ID.
Public
mimeType
String,
type
If
of
multimedia
error
Note:
if
messageID
occurs, return
NULL.
textContent
Note:
message
format
by
format:
mimeType. If in text/plain, it is
FreePP
plain
Unix
corresponding
application/json, it is JSON
timestamp
string.
(s)-two
filePath
random
digits
distinguishes
string;
if
in
ID
this
MessageID
ID-
If
valid,
re-send
its
message,
NULL.
Note: this is asynchronous
file.
String,
If
ID
for
retransmission
through
corresponding
callback interface.
+
(NSString
*)sendGroupMessage
dstGroupID
String,
unique
identifier
of
Send
String-
multimedia
target group.
message
mimeType
If
String,
type
of
multimedia
ID.
error
one-to-many
message
to
specific group.
Note:
if
messageID
occurs, return
message
NULL.
textContent
Note:
message
format
format:
distinguishes
by
ID
mimeType.
FreePP
filePath
Unix
timestamp
through
file.
(s)-two
callback interface.
String,
message
ID
for
ID-
random
retransmission.
digits
CALLBACK
messageID
No
of
(void)onSendMessageEvent
sendResult
code table.
For
one-to-one
both
text
and
and
group
file
message,
Public
onUploadMessageAttachmen
tProgressEvent will be sent
first for callback response.
CALLBACK
messageID
(void)onDownloadMessage
be downloaded
AttachmentEvent
downloadResult
App
Integer,
No
message
download
Callback
interface
updates
of
message
interface
msgType
No
Integer,
(void)onReceiveMessageEv
including:
function
ent
One-to-one/group message
App
message
type
Message
of
message,
receiving
state
interface,
unifies
to
it
is
receive
through
this
callback function.
notification
msgObject
Message object, defined by
each platform, it varies with
different message type. (Note:
refer
to
message
object
comparison table.)
+ (void)clearMessage
messageIDArray
String
array,
No
identifier
of
message
successfully,
this
userInfo
(void)receivedPushNotifica
Dictionary
tionInfo
system
No
type,
Push
received
notification,
(void)application:
(UIApplication *)application
local notifications.
didReceiveRemoteNotificatio
n:(NSDictionary
*)userInfo
NS_AVAILABLE_IOS(3_0);
-
(void)application:
(UIApplication *)application
didReceiveLocalNotification:
Public
(UILocalNotification
*)notification
NS_AVAILABLE_IOS(4_0);
Call in the above system
delegate method, and pass in
userInfo
or
notification.userInfo to this
interface as parameters.
+ (int)reportMessageStatus
dstFreePPID
Integer-
whether send
messageID
successfully.
0-
1 = Delivered
-1-no
2 = Read
network,
unable
to
send.
type
Attribute
(int)
0
NSString * mimeType
NSString * srcFreePPID
NSString * dstFreePPID
NSString * textContent
NSString * messageID
NSString * filePath
int createTime
int sessionType
GroupInfo * freeppGroup
NSString * groupID
NSString * groupName
NSString * invitorFreePPID
NSArray * arrayJoinFreePPID
int createTime
NSString * groupID
NSArray * arrayLeaveFreePPID
int leaveReason
int createTime
Message
JoinGroupMessage
LeaveGroupMessage
Public
3(message
receiving
state
notification)
NSString * messageID
int messageStatus
int createTime
MessageStatus
filePath: string, used inside SDK, file path when sending file, invalid when receiving
message;
freeppGroup: GroupInfo object pointer, group message object, valid for group chat;
leaveReason: integer, reason for leaving group (=0 kicked out; =1 leave).
Parameter
Return value
Notice
messageID
Integer-0: added to
Download
(int)downloadMess
download
ageAttachment
required to be downloaded.
for
isThumbnail
code table.
notification
other
queue,
error
attachment
message.
OnReceivedMessageevent
from
Public
image( Note: invalid for non-
interface,
image type).
downloadMessageAttachment
filePath
interface
String,
local
file
path
of
then
to
calls
download
Note:
name.
this is asynchronous
be
returned
to
APP
messageID
No
(void)onDownload
download
MessageAttachme
downloadProgress
ntProgressEvent
Integer,
percentage of current
Callback
interface
of
progress,
file
it
is
No
(void)onUploadMe
message;
ssageAttachmentP
uploadProgress
method interface.
rogressEvent
Integer,
percentage of current
Callback
interface
of
file
or
sendGroupMessage interface.
Parameter
Return value
Notice
groupName
Create
group.
Invite
(NSString
*)createGroupChat
new
the group.
memberArray
String array, FreePP ID
list of group member.
+ (int)joinGroupChat
groupID
String, group ID returned
join
group.
memberArray
The
client
receive
user to
specific
invited
will
Public
member.
notification
by
message.
+
groupID
Leave
specific
(int)leaveGroupChat
group, or kick
memberArray
of group.
client
member.
receive
will
notification
by
message.
+
groupID
Get information
(FreePPGroupObject
of
*)getGroupChatInfo
GroupInfo)
group
group;
message.
arrayMemberFreePPID: string
array,
FreePP
ID
of
group
member;
Parameter
Return value
Notice
String-calls
Initiate
called user.
session
(call-id).
(NSString
*)makeCall
identifier
free
FreePP
call.
1= Audio call
2= Video call
5=Full duplex audio channel for
push-to-talk
+
(NSString
String-calls
specific
Public
*)makeOutbound
callee.
session
Call
viaOutbound
(call-id).
identifier
audio call.
(NSString
*)joinConference
dstIDs:
FreePP
ID
of
participants.
media:
String-session
identifier
integer,
conferences
media type.
server: string,
conference
repeatedly,
(conference-id).
room ID of
for
inviting
same conference.
of participants.
Or
4=video
conference
call.
+
(int)
getNetworkQualit
Integer-network
Get
of specific call.
transmission
[0, 6].
call.
Integer-whether
score
of
network
quality
of
(reserve)
+
callId:
string, calls
session
(int)hangupCall
operate
or onReceiveCallEvent)
successfully.
+(int)
conferenceId:
hangupConferen
ce
string, session
Integer-whether
operate
(returned by joinConference or
successfully.
conference room)
Integer-whether
operate
onReceiveCallEvent)
+ (int)answerCall
callId:
identifier
+ (int)holdCall
(returned
by
onReceivedCall)
successfully.
callId:
Integer-whether
identifier.
operate
resume call.
successfully.
(0) call.
+ (int)muteCall
Integer-whether
operate
unmute microphone.
identifier.
successfully.
callID:
Integer-whether
identifier
operate
the
successfully.
number to be sent.
peer
user
secondary dialing.
of
an
Public
interval for
sending multiple key digits,
unit: ms (Note: if send a
digit only, it can be set as 0).
time: integer,
Parameter
CALLBACK
callId:
identifier
(void)onReceiveCa
conference.
llEvent
string,
of
session
call
Return value
Notice
No
or
of caller.
callerName:
string,
PUSH
notification
caller.
automatically;
of call request.
1= Audio call;
incoming
2= Video call;
notification
callId:
string,
identifier
(void)onCallStateE
conference.
vent
of
session
call
or
No
display
different
prompts:
1) IDLENo
1 - INITIATE
2 - ANSWER
3 - HOLD
callee)
Public
4 - HANGUP
3) ANSWER- Talking
reason: integer.
4) HOLD
5) HANGUP
connection
conferenceId:
string,
Notify
conferences
(void)onConferenc
identifier.
eStateEvent
APP
of
certain
session
No
identifier of participants.
should
display
different
of participants.
prompts:
values:
callee)
1 - INITIATE
2) ANSWER
2 - ANSWER
3) HANGUP
3 - HANGUP
When state is HANGUP,
identify reason for finishing
a call, including: user offline,
network
connection
(void)
of participant.
No
state change.
onRemoteVideoSta
teEvent
should
values:
prompts:
0 - STOP
display
different
Public
1 - START
video sending
2) STOP Stop remote
video sending
Parameter
+ (int)setCamera
device:
integer,
device
Return value
Notice
Integer-whether
identifier, e.g.:
operate
0 - Rear camera
successfully.
1 - Front camera
-1 Close preview
+
device:
integer,
speaker
Integer-whether
(int)setAudioOutp
operate
ut
0 - Receiver
successfully.
1 - Speaker
2
-Bluetooth
headset
connected currently
+
displayRemote:
(int)setVideoDispla
y
View
Integer-whether
operate
in video call.
successfully.
View
ringbackFilePath:
(int)setRingbackTo
ne
string,
Integer-whether
operate
Supported
successfully.
pcm(16K) wav(pcm:8k,16k,32k,11
file
format:
025,22050,44100k,48k;pcma;pcmu)
Parameter
Return value
Notice
No
(NSArray
Public
*)getAllCallIDList
+
(FreePPCallInfo
*)getCallInfo
all callIDs.
callId:
FreePPCallInfo
string,
object
session
information object.
identifier of
call
pointer-call
attributes:
callId: string, session identifier of call or
or
conference.
conference.
Parameter
Return
Notice
value
+ (int)
Integer-
Subscribe
subConferenceVideo
of video publisher
whether
operate
attendee)
successfully
+(int)listConference
conferenceID:
string,
conference room ID
Integer-
whether
operate
successfully
will
report
participant
state
conferenceID:
ctrlConference
string,
Integer-
conference room ID
whether
operate
action to be performed
successfully
Public
FreePPID:
string,
FreePP ID of participant
controlled
Para
Return value
Notice
Integer-
mete
r
+ (int)
No
startVideoSend
whether
operate
Note:
successfully
+(int)
No
Integer-
stopVideoSend
whether
operate
successfully
file
message,get
file
upload
progress
through
Public
callback function;
Step 6. For file message, receiver calls downloadMessageAttachment function to download
attachment of the message;
Step7.
Receiver
gets
file
download
progress
through
For
file
message,
get
file
upload
progress
through
onUploadMessageAttachmentEventcallback function;
Step 5. Receiver will receive message or notification through onReceivedMessageEvent
callback function;
Step 6. For file message, receiver calls downloadMessageAttachment function to download
attachment of the message;
Step7.
Receiver
gets
file
download
progress
through
onDownloadMessageAttachmentEvent callback function, and gets state value of
downloaded message file through onDownloadMessageEvent callback function.
Public
Public
Public
Public
Public
Public
Public
Meaning
No network connection
-2
Related to user
-10
Illegitimate AppKey
-11
Parameter error
-12
Repeat login, current user is kicked out, or already logs in on other device
-13
-14
-15
-16
-17
Related to message
-20
-21
Public
-22
-31
-32
-41
-42
-43
-44
-45
-51
No current conference
-52
Public