AudioUnit实战

发表于   |  

前面学习了AudioUnit相关理论姿势，现在结合实际demo继续深入学习一下。。。

AudioUnit–播放本地音频

因为AudioUnit只支持PCM文件，所以这里我们先使用一个本地的PCM文件进行播放，后续会有播放其他格式文件的demo,需要用到转码的相关API。

这里我们使用NSInputStream直接读取音频文件，因为不需要转码，所以读取到的数据就是可以直接拿来播放的数据。

1
2
3
4
5
6
7
8
9
10
NSInputStream *inputStream;
 	NSURL *url = [NSURL fileURLWithPath:self.pcmPath];
   inputStream = [NSInputStream inputStreamWithURL:url];
   if (!inputStream) {
       NSLog(@"打开文件失败!!!!%@",url);
   }
   else{
       [inputStream open];
   }

初始化AudioUnit相关内容前，我们需要全局设置一下AVAudioSession

1
2
3
4
5
//设置audiosession
   NSError *error = nil;   
   AVAudioSession *audioSession = [AVAudioSession sharedInstance];
   [audioSession setCategory:AVAudioSessionCategoryPlayback error:&error];

然后可以开始初始化AudioUnit，这里我们需要的是Output type

1
2
3
4
5
6
7
8
9
10
11
12
13
   AudioUnit playerAudioUnit;
//AU描述
   AudioComponentDescription audioUnitDesc;
   audioUnitDesc.componentType = kAudioUnitType_Output;
   audioUnitDesc.componentSubType = kAudioUnitSubType_RemoteIO;
   audioUnitDesc.componentManufacturer = kAudioUnitManufacturer_Apple;
   audioUnitDesc.componentFlags = 0;
   audioUnitDesc.componentFlagsMask = 0;
   
   //AudioUnit裸创建
   AudioComponent audioComponent = AudioComponentFindNext(NULL, &audioUnitDesc);
   AudioComponentInstanceNew(audioComponent, &playerAudioUnit);

接下来有一个kAudioOutputUnitProperty_EnableIO属性的设置，这里如果只是播放功能的话，实际上是不需要设置的，因为AudioUnit默认就是支持播放的，如果需要输入功能（比如录音）时，就必须设置这个属性，之后的demo会有使用，这里我们先不设置这个属性，设置的相关格式如下：

1
2
3
4
5
6
7
8
//通用参数设置,这里是设置扬声器
  OSStatus status = noErr;
  UInt32 flag = 1;
  UInt32 outputBus = 0;//Element 0
  status = AudioUnitSetProperty(playerAudioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Output, outputBus, &flag, sizeof(flag));
  if (status) {
      NSLog(@"AudioUnitSetProperty error with status:%d", status);
  }

然后我们需要设置AudioUnit的播放音频的格式，这里一般都是PCM格式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//设置音频具体结构
   AudioStreamBasicDescription audioStreamDesc;
//    bzero(&audioStreamDesc, sizeof(audioStreamDesc));
   memset(&audioStreamDesc, 0, sizeof(audioStreamDesc));
   audioStreamDesc.mFormatID = kAudioFormatLinearPCM;
   audioStreamDesc.mSampleRate = 44100;//采样率
   audioStreamDesc.mChannelsPerFrame = 1;//声道数
   audioStreamDesc.mFramesPerPacket = 1;//每帧只有一个packet
   audioStreamDesc.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger;//kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;
   audioStreamDesc.mBitsPerChannel = 16;//位深
   audioStreamDesc.mBytesPerFrame = 2;
   audioStreamDesc.mBytesPerPacket = 2;
//这里是设置的输出音频的格式
   status = AudioUnitSetProperty(playerAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, outputBus, &audioStreamDesc,sizeof(audioStreamDesc));
   if (status) {
       NSLog(@"AudioUnitSetProperty error with status:%d", status);
   }

最后设置一下输出的回调，这个回调是每次扬声器需要播放数据时都会调一次这个回调，然后我们在这个回调里面把要播放的数据塞给扬声器。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
	//callback
    AURenderCallbackStruct callbackStruct;
    callbackStruct.inputProc = PlayCallback;
    callbackStruct.inputProcRefCon = (__bridge void *)self;
    AudioUnitSetProperty(playerAudioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, outputBus, &callbackStruct, sizeof(callbackStruct));
	//最后初始化AudioUnit
    OSStatus result = AudioUnitInitialize(playerAudioUnit);
    NSLog(@"result = %d",result);
    
    //回调实现
    static OSStatus PlayCallback(void *inRefCon,AudioUnitRenderActionFlags *ioActionFlag,const AudioTimeStamp *inTimeStamp,UInt32 inBusNumber,UInt32 inNumberFrames,AudioBufferList *ioData){
    __unsafe_unretained LLYAudioUnitPlayer *play = (__bridge LLYAudioUnitPlayer *)inRefCon;
    ioData->mBuffers[0].mDataByteSize = (UInt32)[play->inputStream read:ioData->mBuffers[0].mData maxLength:(NSInteger)ioData->mBuffers[0].mDataByteSize];
    NSLog(@"out size: %d", ioData->mBuffers[0].mDataByteSize);
    if (ioData->mBuffers[0].mDataByteSize <= 0) {
        dispatch_async(dispatch_get_main_queue(), ^{
            [play stop];
        });
    }
    return noErr;
}

然后就可以开始播放音频了

1
2
3
4
5
6
7
8
9
10
11
12
- (void)play{
    
    [self initAudioUnit];
    
    AudioOutputUnitStart(playerAudioUnit);
}
- (void)stop{
    
    AudioOutputUnitStop(playerAudioUnit);
    [inputStream close];
}

AudioUnit–录音+播放+保存本地

这个demo做的事情是把本地播放的音频和录制的音频一起输出并写入文件。播放音频部分和上面的流程一样，这里主要说一下录制音频部分的流程。

AVAudioSession的设置，这里我们需要用到录音功能，同时我们还需要设置一下采样频率。

1
2
3
4
5
6
7
8
9
10
//设置AVAudioSession
   [[AVAudioSession sharedInstance]setCategory:AVAudioSessionCategoryPlayAndRecord error:&error];
   if (error) {
       NSLog(@"setCategory error %@",error);
   }
   
   [[AVAudioSession sharedInstance] setPreferredIOBufferDuration:0.05 error:&error];
   if (error) {
       NSLog(@"setPreferredIOBufferDuration error:%@", error);
   }

我们使用AudioBufferList 这个数据结构来保存获取到的录音音频数据，先对该数据结构做一个初始化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
AudioBufferList *bufferList;
// buffer list
uint32_t numberBuffers = 2;
bufferList = (AudioBufferList *)malloc(sizeof(AudioBufferList) + (numberBuffers - 1) * sizeof(AudioBuffer));
bufferList->mNumberBuffers = numberBuffers;
bufferList->mBuffers[0].mNumberChannels = 1;
bufferList->mBuffers[0].mDataByteSize = BUFFERSIZE;
bufferList->mBuffers[0].mData = malloc(BUFFERSIZE);
for (int i =1; i < numberBuffers; ++i) {
    bufferList->mBuffers[i].mNumberChannels = 1;
    bufferList->mBuffers[i].mDataByteSize = BUFFERSIZE;
    bufferList->mBuffers[i].mData = malloc(BUFFERSIZE);
}

接着上个demo的代码，这里我们再设置一下输入的数据结构

1
2
3
4
5
6
7
8
9
10
11
12
AudioStreamBasicDescription outputFormat = inputFormat;
   outputFormat.mChannelsPerFrame = 2;
   status = AudioUnitSetProperty(recordAudioUnit,
                                 kAudioUnitProperty_StreamFormat,
                                 kAudioUnitScope_Input,
                                 OUTPUT_BUS,
                                 &outputFormat,
                                 sizeof(outputFormat));
   
   if (status != noErr) {
       NSLog(@"AudioUnitGetProperty error, ret: %d", status);
   }

然后我们需要设置AudioUnit的输入能力

1
2
3
4
5
6
7
8
9
10
11
// enable record
  UInt32 flag = 1;
  status = AudioUnitSetProperty(recordAudioUnit,
                                kAudioOutputUnitProperty_EnableIO,
                                kAudioUnitScope_Input,
                                INPUT_BUS,
                                &flag,
                                sizeof(flag));
  if (status != noErr) {
      NSLog(@"AudioUnitGetProperty error, ret: %d", status);
  }

之后设置一下录音的回调，在回调中获取录音的音频数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
    // set callback
    AURenderCallbackStruct recordCallback;
    recordCallback.inputProc = RecordCallback;
    recordCallback.inputProcRefCon = (__bridge void *)self;
    status = AudioUnitSetProperty(recordAudioUnit,
                                  kAudioOutputUnitProperty_SetInputCallback,
                                  kAudioUnitScope_Output,
                                  INPUT_BUS,
                                  &recordCallback,
                                  sizeof(recordCallback));
    if (status != noErr) {
        NSLog(@"AudioUnitGetProperty error, ret: %d", status);
    }
    
    
    #pragma mark - callback
    
static OSStatus RecordCallback(void *inRefCon,
                               AudioUnitRenderActionFlags *ioActionFlags,
                               const AudioTimeStamp *inTimeStamp,
                               UInt32 inBusNumber,
                               UInt32 inNumberFrames,
                               AudioBufferList *ioData)
{
    LLYAudioUnitRecord *record = (__bridge LLYAudioUnitRecord *)inRefCon;
    record->bufferList->mNumberBuffers = 1;
    OSStatus status = AudioUnitRender(record->recordAudioUnit, ioActionFlags, inTimeStamp, inBusNumber, inNumberFrames, record->bufferList);
    if (status != noErr) {
        NSLog(@"AudioUnitRender error:%d", status);
    }
    
    NSLog(@"size1 = %d", record->bufferList->mBuffers[0].mDataByteSize);
    [record writePCMData:record->bufferList->mBuffers[0].mData size:record->bufferList->mBuffers[0].mDataByteSize];
    
    return noErr;
}

这里我们把录音的音频数据保存在了bufferList中。这个bufferList中的数据将被手动塞给播放的回调进行播放，播放（输出）的回调代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
static OSStatus OutPlayCallback(void *inRefCon,
                             AudioUnitRenderActionFlags *ioActionFlags,
                             const AudioTimeStamp *inTimeStamp,
                             UInt32 inBusNumber,
                             UInt32 inNumberFrames,
                             AudioBufferList *ioData) {
    LLYAudioUnitRecord *record = (__bridge LLYAudioUnitRecord *)inRefCon;
    memcpy(ioData->mBuffers[0].mData, record->bufferList->mBuffers[0].mData, record->bufferList->mBuffers[0].mDataByteSize);
    ioData->mBuffers[0].mDataByteSize = record->bufferList->mBuffers[0].mDataByteSize;
    
    NSInteger bytes = BUFFERSIZE < ioData->mBuffers[1].mDataByteSize * 2 ? BUFFERSIZE : ioData->mBuffers[1].mDataByteSize * 2; //
    bytes = [record->inputStream read:record->buffer maxLength:bytes];
    
    for (int i = 0; i < bytes; ++i) {
        ((Byte*)ioData->mBuffers[1].mData)[i/2] = record->buffer[i];
    }
    ioData->mBuffers[1].mDataByteSize = (UInt32)bytes / 2;
    
    if (ioData->mBuffers[1].mDataByteSize < ioData->mBuffers[0].mDataByteSize) {
        ioData->mBuffers[0].mDataByteSize = ioData->mBuffers[1].mDataByteSize;
    }
    
    
    NSLog(@"size2 = %d", ioData->mBuffers[0].mDataByteSize);
    
    return noErr;
}

这里我们给ioData->mBuffers塞了2组数据，一组是从bufferList中获取的录音音频数据，另外一组是从inputStream中读取的本地音频数据，所以播放时既能听到本地音频也能听到我们录音的音频。

将音频写入文件的代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
- (void)writePCMData:(Byte *)buffer size:(int)size {
    static FILE *file = NULL;
    NSString *pathStr = [LLYAudioUnitRecord recordPath];
    if (!file) {
        file = fopen(pathStr.UTF8String, "w");
    }
    fwrite(buffer, size, 1, file);
}
+ (NSString *)recordPath{
    return [CommonUtil documentsPath:@"/record.pcm"];
}

AudioUnit–播放MP3

上面说过，AudioUnit是不支持直接播放MP3文件格式的，所以这里我们先把MP3转码为pcm格式然后再塞给播放回调去播放。

播放的逻辑我们已经很清楚了，这里我主要说一下怎么转码。这里我们用到了AudioFile和AudioConverter两个类，AudioFile主要做音频相关信息的读取，AudioConverter主要做转码的工作。

先来看一下AudioFile的相关代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
   AudioFileID audioFileID;
   AudioStreamBasicDescription audioFileFormat;
   AudioStreamPacketDescription *audioPacketFormat;
   
SInt64 readedPacket;
   UInt64 packetNums;
   UInt64 packetNumsInBuffer;
   
   //获取源音频的ID
   NSString *mp3Path = [CommonUtil bundlePath:@"abc.mp3"];
   NSURL *mp3Url = [NSURL fileURLWithPath:mp3Path];
   OSStatus status = AudioFileOpenURL((__bridge CFURLRef)mp3Url, kAudioFileReadPermission, 0, &audioFileID);
   if (status) {
       NSLog(@"打开文件失败 %@", mp3Url);
       return ;
   }
   
   //获取源音频的FileFormat
   uint32_t size = sizeof(AudioStreamBasicDescription);
   status = AudioFileGetProperty(audioFileID, kAudioFilePropertyDataFormat, &size, &audioFileFormat);
   if (status) {
       NSLog(@"获取fileformat失败 error status %d", status);
       return ;
   }
   //获取源音频的packetnum
   size = sizeof(packetNums);
   status = AudioFileGetProperty(audioFileID, kAudioFilePropertyAudioDataPacketCount, &size, &packetNums);
   if (status) {
       NSLog(@"获取packetnum失败 error status %d", status);
       return ;
   }
   
   //获取源音频单个packet的最大buffer数
   uint32_t sizePerPacket = audioFileFormat.mFramesPerPacket;
   if (sizePerPacket == 0) {
       size = sizeof(sizePerPacket);
       status = AudioFileGetProperty(audioFileID, kAudioFilePropertyMaximumPacketSize, &size, &sizePerPacket);
       if (status) {
           NSLog(@"获取packetmaxnum失败 error status %d", status);
           return ;
       }
   }
   
   //获取源音频的packetformat
   audioPacketFormat = malloc(sizeof(AudioStreamPacketDescription) * (CONST_BUFFER_SIZE/sizePerPacket + 1));

这里说一下packetnum这个属性，因为音频数据流都是以一个个packet的格式封装起来的，所以这个获取的packetnum实际上就是音频的总长度，而每个packet里面又以帧（frame）为单位做了一层封装，每个packet里面的帧数不固定。每一帧就是实际的音频数据了。

因为数据的处理都在播放回调中，我们来看看AudioConverter都做了哪些工作：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
	//上面拿到了音频的数据格式，这里用来做AudioConverter的初始化
	//初始化audioconverter
    status = AudioConverterNew(&audioFileFormat, &outputFormat, &audioConverter);
    if (status) {
        NSLog(@"AudioConverterNew eror with status:%d", status);
    }
    
	OSStatus ConPlayCallback(void *inRefCon,
                      AudioUnitRenderActionFlags *ioActionFlags,
                      const AudioTimeStamp *inTimeStamp,
                      UInt32 inBusNumber,
                      UInt32 inNumberFrames,
                      AudioBufferList *ioData) {
    LLYAudioUnitConverter *converter = (__bridge LLYAudioUnitConverter *)inRefCon;
    
    converter->bufferList->mBuffers[0].mDataByteSize = CONST_BUFFER_SIZE;
    OSStatus status = AudioConverterFillComplexBuffer(converter->audioConverter, InputDataProc, inRefCon, &inNumberFrames, converter->bufferList, NULL);
    if (status) {
        NSLog(@"转换格式失败 %d", status);
    }
    
    NSLog(@"out size: %d", converter->bufferList->mBuffers[0].mDataByteSize);
    memcpy(ioData->mBuffers[0].mData, converter->bufferList->mBuffers[0].mData, converter->bufferList->mBuffers[0].mDataByteSize);
    ioData->mBuffers[0].mDataByteSize = converter->bufferList->mBuffers[0].mDataByteSize;
    
    //    fwrite(player->buffList->mBuffers[0].mData, player->buffList->mBuffers[0].mDataByteSize, 1, [player pcmFile]);
    
    if (converter->bufferList->mBuffers[0].mDataByteSize <= 0) {
        dispatch_async(dispatch_get_main_queue(), ^{
            [converter onPlayEnd];
        });
    }
    return noErr;
}

可以看到，是调用了AudioConverterFillComplexBuffer这个方法做的转码，其中第二个参数又是一个回调函数，转码后的数据同样是保存在converter->bufferList这个结构里面，然后在后面塞给ioData->mBuffers去播放。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
OSStatus InputDataProc(AudioConverterRef inAudioConverter, UInt32 *ioNumberDataPackets, AudioBufferList *ioData, AudioStreamPacketDescription **outDataPacketDescription, void *inUserData)
{
    LLYAudioUnitConverter *player = (__bridge LLYAudioUnitConverter *)(inUserData);
    
    UInt32 byteSize = CONST_BUFFER_SIZE;
    OSStatus status = AudioFileReadPacketData(player->audioFileID, NO, &byteSize, player->audioPacketFormat, player->readedPacket, ioNumberDataPackets, player->convertBuffer);
    
    if (outDataPacketDescription) { // 这里要设置好packetFormat，否则会转码失败
        *outDataPacketDescription = player->audioPacketFormat;
    }
    
    if(status) {
        NSLog(@"读取文件失败");
    }
    
    if (!status && ioNumberDataPackets > 0) {
        ioData->mBuffers[0].mDataByteSize = byteSize;
        ioData->mBuffers[0].mData = player->convertBuffer;
        player->readedPacket += *ioNumberDataPackets;
        return noErr;
    }
    else {
        return NO_MORE_DATA;
    }
}

这个回调函数主要负责读取mp3文件给上面的AudioConverterFillComplexBuffer函数用来做转码处理。

AudioUnit–更简单的播放MP3

上一个demo中我们使用AudioUnit播放了一个MP3文件，用到了AudioFile和AudioConverter类的相关方法，可以看到，使用这种方式播放MP3文件是比较繁琐的，我们要获取音频的很多信息，要设置回调，要使用一系列的相关api。。。下面介绍一种更简单的播放MP3的方式，使用ExtAudioFile，看名字就知道，它是上面介绍的AudioFile的扩展。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
//ExtAudioFileRef对象
 ExtAudioFileRef extAudioFile;
   AudioStreamBasicDescription audioFileFormat;
   AudioStreamBasicDescription outputFormat;
   
    OSStatus status = noErr;
   NSURL *mp3Url = [NSURL fileURLWithPath:[CommonUtil bundlePath:@"/abc.mp3"]];
   status = ExtAudioFileOpenURL((__bridge CFURLRef)mp3Url, &extAudioFile);
   if (status) {
       NSLog(@"打开文件失败");
   }
   
   //获取MP3音频的格式（非必需）
   uint32_t size = sizeof(AudioStreamBasicDescription);
   status = ExtAudioFileGetProperty(extAudioFile, kExtAudioFileProperty_FileDataFormat, &size, &audioFileFormat);
   if (status) {
       NSLog(@"读取音频格式失败");
   }
   
    //initFormat 设置输出格式
   memset(&outputFormat, 0, sizeof(outputFormat));
   outputFormat.mSampleRate       = 44100;
   outputFormat.mFormatID         = kAudioFormatLinearPCM;
   outputFormat.mFormatFlags      = kLinearPCMFormatFlagIsSignedInteger;
   outputFormat.mBytesPerPacket   = 2;
   outputFormat.mFramesPerPacket  = 1;
   outputFormat.mBytesPerFrame    = 2;
   outputFormat.mChannelsPerFrame = 1;
   outputFormat.mBitsPerChannel   = 16;
    status = ExtAudioFileSetProperty(extAudioFile, kExtAudioFileProperty_ClientDataFormat, size, &outputFormat);
   if (status) {
       NSLog(@"设置转码音频格式失败");
   }
   
   //获取总长度（非必需）
    size = sizeof(totalFrame);
   status = ExtAudioFileGetProperty(extAudioFile, kExtAudioFileProperty_FileLengthFrames, &size, &totalFrame);
   if (status) {
       NSLog(@"获取音频总长度失败");
   }
   NSLog(@"音频总长度：%llu",totalFrame);

AudioUnit相关的代码参考上面的几个demo. 这个必需的操作就是open file，获取到extAudioFile。然后我们看看回调里面做了什么

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
OSStatus ExtPlayCallback(void *inRefCon,
                      AudioUnitRenderActionFlags *ioActionFlags,
                      const AudioTimeStamp *inTimeStamp,
                      UInt32 inBusNumber,
                      UInt32 inNumberFrames,
                      AudioBufferList *ioData) {
    LLYAudioUnitExtPlayer *player = (__bridge LLYAudioUnitExtPlayer *)inRefCon;
    player->bufferList->mBuffers[0].mDataByteSize = BUFFER_SIZE_CONST;
    OSStatus status = ExtAudioFileRead(player->extAudioFile, &inNumberFrames, player->bufferList);
    NSLog(@"inNumberFrames = %d",inNumberFrames);
    if (status) {
        NSLog(@"转码失败");
    }
    if (!inNumberFrames) {
        NSLog(@"播放结束");
    }
    NSLog(@"out size : %d",player->bufferList->mBuffers[0].mDataByteSize);
    memcpy(ioData->mBuffers[0].mData, player->bufferList->mBuffers[0].mData, player->bufferList->mBuffers[0].mDataByteSize);
    ioData->mBuffers[0].mDataByteSize = player->bufferList->mBuffers[0].mDataByteSize;
    
    player->readedFrame += player->bufferList->mBuffers[0].mDataByteSize / player->outputFormat.mBytesPerFrame;
    
    NSLog(@"readedFrame = %d",player->readedFrame);
    
    if (player->bufferList->mBuffers[0].mDataByteSize <= 0) {
        dispatch_async(dispatch_get_main_queue(), ^{
            [player onPlayEnd];
        });
    }
    return noErr;
}

这里我们只调用了一个ExtAudioFileRead方法，就把MP3文件转成了pcm文件存到了player->bufferList中，最终塞给ioData->mBuffers去播放即可。是不是比上面demo要简单很多呢。

AudioUnit–使用AUGraph播放伴奏+录音

前面都是直接使用AudioUnit实现播放，录音相关工作，这一小节我们来学习一下使用AUGraph管理AudioUnit，当你需要多个AudioUnit来实现不同的模块，且各模块的输入输出直接又有相关联时，使用AUGraph是比较好的选择，因为它可以为AudioUnit之间建立连接，直接进行数据传递，省去了我们手动传递数据的麻烦。

这里我是分了三步来实现整个功能的：

  • 1.先播放本地音频
  • 2.加入录音功能
  • 3.将录音音频和本地音频送给mix,mix的输出绑定到io的输出

首先来看第一步，使用AUGraph播放本地音频

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
//相关变量
   AUGraph playerGraph;
   
   //io相关
   AUNode ioNode;
   AudioUnit ioUnit;
   
   AudioStreamBasicDescription audioFormat;
   
   NSInputStream *inputStream;
//init
    
   NSURL *url = [NSURL fileURLWithPath:[CommonUtil bundlePath:@"/abc.pcm"]];
   inputStream = [NSInputStream inputStreamWithURL:url];
   if (!inputStream) {
       NSLog(@"打开文件失败!!!!%@",url);
   }
   else{
       [inputStream open];
   }
   
   AVAudioSession *audioSession = [AVAudioSession sharedInstance];
   [audioSession setCategory:AVAudioSessionCategoryPlayback error:nil]; // 只有播放
OSStatus status = noErr;
   status = NewAUGraph(&playerGraph);
   CheckStatus(status, @"创建AUGraph失败", YES);
   
   AudioComponentDescription outputAudioDesc;
   outputAudioDesc.componentType = kAudioUnitType_Output;
   outputAudioDesc.componentSubType = kAudioUnitSubType_RemoteIO;
   outputAudioDesc.componentManufacturer = kAudioUnitManufacturer_Apple;
   outputAudioDesc.componentFlags = 0;
   outputAudioDesc.componentFlagsMask = 0;
   
   status = AUGraphAddNode(playerGraph, &outputAudioDesc, &ioNode);
   CheckStatus(status, @"绑定node失败", YES);
   
   status = AUGraphOpen(playerGraph);
   CheckStatus(status, @"打开AUGraph失败", YES);
   
   status = AUGraphNodeInfo(playerGraph, ioNode, NULL, &ioUnit);
   CheckStatus(status, @"创建AudioUnit失败", YES);
   // audio format
   audioFormat.mSampleRate = 44100;
   audioFormat.mFormatID = kAudioFormatLinearPCM;
   audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsNonInterleaved;
   audioFormat.mFramesPerPacket = 1;
   audioFormat.mChannelsPerFrame = 1;
   audioFormat.mBytesPerPacket = 2;
   audioFormat.mBytesPerFrame = 2;
   audioFormat.mBitsPerChannel = 16;
   status = AudioUnitSetProperty(ioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &audioFormat, sizeof(audioFormat));
   CheckStatus(status, @"设置输入音频数据格式失败", YES);
   status = AudioUnitSetProperty(ioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 1, &audioFormat, sizeof(audioFormat));
   CheckStatus(status, @"设置输出音频格式失败", YES);
   
   //callback
   AURenderCallbackStruct callbackStruct;
   callbackStruct.inputProc = LLYPlayCallback;
   callbackStruct.inputProcRefCon = (__bridge void *)self;
   AudioUnitSetProperty(playerUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, 0, &callbackStruct, sizeof(callbackStruct));
   
   status  = AUGraphInitialize(playerGraph);
   CheckStatus(status, @"Graph初始化失败", YES);
   status = AUGraphStart(playerGraph);
   CheckStatus(status, @"Graph start失败", YES);

io的回调函数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
	static OSStatus LLYPlayCallback(void *inRefCon,AudioUnitRenderActionFlags *ioActionFlag,const AudioTimeStamp *inTimeStamp,UInt32 inBusNumber,UInt32 inNumberFrames,AudioBufferList *ioData){
    
    __unsafe_unretained LLYAUGraphRecord *play = (__bridge LLYAUGraphRecord *)inRefCon;
    
    ioData->mBuffers[0].mDataByteSize = (UInt32)[play->inputStream read:ioData->mBuffers[0].mData maxLength:(NSInteger)ioData->mBuffers[0].mDataByteSize];
    NSLog(@"out size: %d", ioData->mBuffers[0].mDataByteSize);
    
    if (ioData->mBuffers[0].mDataByteSize <= 0) {
        dispatch_async(dispatch_get_main_queue(), ^{
            [play stop];
        });
    }
    return noErr;
}

第二步：加入录音功能，这里其实只需要对第一步做一下修改,将io回调数据处理的bus改为输入，设置输入相关属性即可

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
   AudioBufferList *buffList;
   Byte *buffer;
   
   
   //init
   [audioSession setCategory:AVAudioSessionCategoryPlayAndRecord error:nil];
   [audioSession setPreferredIOBufferDuration:0.02 error:&error];
   
   // buffer
   uint32_t numberBuffers = 1;
   buffList = (AudioBufferList *)malloc(sizeof(AudioBufferList));
   buffList->mNumberBuffers = numberBuffers;
   buffList->mBuffers[0].mNumberChannels = 1;
   buffList->mBuffers[0].mDataByteSize = GRAPH_CONST_BUFFER_SIZE;
   buffList->mBuffers[0].mData = malloc(GRAPH_CONST_BUFFER_SIZE);
   buffer = malloc(GRAPH_CONST_BUFFER_SIZE);
//设置输入格式
   status = AudioUnitSetProperty(ioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &audioFormat, sizeof(AudioStreamBasicDescription));
   CheckStatus(status, @"设置输出音频格式失败", YES);
   
   
//设置输入部分的回调，实际的录音数据是在这里获取的，
   AURenderCallbackStruct outputCallbackStruct;
   outputCallbackStruct.inputProc = OutputPlayCallback;
   outputCallbackStruct.inputProcRefCon = (__bridge void *)self;
   status = AudioUnitSetProperty(ioUnit, kAudioOutputUnitProperty_SetInputCallback, kAudioUnitScope_Output, 1, &outputCallbackStruct, sizeof(outputCallbackStruct));
   CheckStatus(status, @"ioUnit绑定回调失败", YES);

io的回调处理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
	static OSStatus OutputPlayCallback(void *inRefCon,AudioUnitRenderActionFlags *ioActionFlag,const AudioTimeStamp *inTimeStamp,UInt32 inBusNumber,UInt32 inNumberFrames,AudioBufferList *ioData){
    
    __unsafe_unretained LLYAUGraphRecord *play = (__bridge LLYAUGraphRecord *)inRefCon;
    
    play->buffList->mNumberBuffers = 1;
    OSStatus status = AudioUnitRender(play->ioUnit, ioActionFlag, inTimeStamp, inBusNumber, inNumberFrames, play->buffList);
    CheckStatus(status, @"获取录音音频失败", YES);
    
//    NSLog(@"record size %d:",play->buffList->mBuffers[0].mDataByteSize);
    
    //将录音写入文件
    
    return noErr;
}

第三步：获取2路音频并同时播放

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
    //负责获取2路音频和混合
    AUNode mixNode;
    AudioUnit mixUnit;
	 //混合相关
    AudioComponentDescription mixAudioDesc;
    mixAudioDesc.componentType = kAudioUnitType_Mixer;
    mixAudioDesc.componentSubType = kAudioUnitSubType_MultiChannelMixer;
    mixAudioDesc.componentManufacturer = kAudioUnitManufacturer_Apple;
    mixAudioDesc.componentFlags = 0;
    mixAudioDesc.componentFlagsMask = 0;
    
    status = AUGraphAddNode(playerGraph, &mixAudioDesc, &mixNode);
    CheckStatus(status, @"绑定混合node失败", YES);
    
    status = AUGraphNodeInfo(playerGraph, mixNode, NULL, &mixUnit);
    CheckStatus(status, @"创建混合AudioUnit失败", YES);
    //绑定nodes，这里就是将mix的输出和io的输出绑定起来，不再需要手动设置io的输出回调了
    status = AUGraphConnectNodeInput(playerGraph, mixNode, 0, ioNode, 0);
    CheckStatus(status, @"绑定node失败", YES);
    //设置bus数
    UInt32 busCount = 2;
    status = AudioUnitSetProperty(mixUnit, kAudioUnitProperty_ElementCount, kAudioUnitScope_Input, 0, &busCount, sizeof(busCount));
    CheckStatus(status, @"设置声道数失败", YES);
    
    //设置混合的输入格式,有多条输入
    //bus0
    status = AudioUnitSetProperty(mixUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &audioFormat, sizeof(audioFormat));
    CheckStatus(status, @"设置混合音频输入数据格式失败", YES);
    //bus1
    status = AudioUnitSetProperty(mixUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 1, &audioFormat, sizeof(audioFormat));
    CheckStatus(status, @"设置混合音频输入数据格式失败", YES);
	 //获取背景音频数据
    AURenderCallbackStruct callback0;
    callback0.inputProc = mixCallback0;
    callback0.inputProcRefCon = (__bridge void *)self;
//    status = AUGraphSetNodeInputCallback(playerGraph, mixNode, 0, &callback0);
    CheckStatus(AudioUnitSetProperty(mixUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, 0, &callback0, sizeof(callback0)), @"add mix callback fail",YES);
    //获取录音数据
    AURenderCallbackStruct callback1;
    callback1.inputProc = mixCallback1;
    callback1.inputProcRefCon = (__bridge void *)self;
//    status = AUGraphSetNodeInputCallback(playerGraph, mixNode, 1, &callback1);
    CheckStatus(AudioUnitSetProperty(mixUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, 1, &callback1, sizeof(callback1)), @"add mix callback fail",YES);

mix的2路数据的回调

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
	//获取本地播放的音频
	static OSStatus mixCallback0(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData) {
    
    __unsafe_unretained LLYAUGraphRecord *play = (__bridge LLYAUGraphRecord *)inRefCon;
    
    ioData->mBuffers[0].mDataByteSize = (UInt32)[play->inputStream read:ioData->mBuffers[0].mData maxLength:(NSInteger)ioData->mBuffers[0].mDataByteSize];
    NSLog(@"audio size: %d", ioData->mBuffers[0].mDataByteSize);
    
    if (ioData->mBuffers[0].mDataByteSize <= 0) {
        dispatch_async(dispatch_get_main_queue(), ^{
            [play stop];
        });
    }
    return noErr;
}
	//获取录音音频
	static OSStatus mixCallback1(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData) {
    
    __unsafe_unretained LLYAUGraphRecord *play = (__bridge LLYAUGraphRecord *)inRefCon;
    memcpy(ioData->mBuffers[0].mData, play->buffList->mBuffers[0].mData, play->buffList->mBuffers[0].mDataByteSize);
    ioData->mBuffers[0].mDataByteSize = play->buffList->mBuffers[0].mDataByteSize;
    
    NSLog(@"record size: %d", ioData->mBuffers[0].mDataByteSize);
    return noErr;
}

在mix的2路回调中，我们分别获取到了数据并塞给mix的输入，然后mix直接输出到io的输出bus进行播放，中间不在需要我们做额外的工作。

我的demo