OS.X and iOS Kernel Programming.md
https://github.com/Apress/os-x-ios-kernel-programming
https://github.com/knightsc/osx_and_ios_kernel_programming

Memory Allocation in Mach

kmem_alloc/kmem_alloc_contig/kmem_free

The I/O Kit framework consists of the kernel level framework, as well as a user space framework called IOKit.framework.
The kernel framework is written in Embedded C++, a subset of C++, whereas the user space framework is C-based.

In the case of a USB-based storage device, it might depent on the IOUSBFamily, as well as
the IOStorageFamily.

In addition to standard C++ runtime, libkern also provides a number of useful classes, the
most fundamental being OSObject, the superclass of every class in I/O kit.

The platform expert itself will form the root node of the tree, IOPlatformExpertDevice.

chapter 3

Xcode and the Kernel Development Environment
The features that are not available include the following:
Exceptions/Multiple inheritance/Templates/Runtime type information

Although both templates create a kernel extension, an I/O Kit driver reuires us to nominate
a hardware device it will match against and will load only if that device is present.
A generic kernel extension, on the other hand, is not a hardware driver and can be loaded
any time by the user
Rather than a compile-time linking, the kernel resolves any library dependencies of a kernel
extension only when the kernel extension is loaded.
Version 9.0.0 corresponds to Mac OS X 10.5.0, the version of the Mac OS X kernel.[uname -r]

chapter 4

The I/O Kit Framework
Kernel.framework/IOKit.framework

The kernel will refuse to load an extension that contains symbols that collide with an
extension that is already loaded, and so to avoid this, Apple recommends that all global
functions, classes, and variables are decorated with a reverse-DNS namming scheme.
Thankfully, the I/O Kit provides a special nub known as IOResources that can be used as
the provider class of a driver that has no hardware device, such as the tutorial driver lised
here. In a system, there will be multiple drivers matching against the IOResources nub, and
so to allow more than one driver to attach itself to IOResources, the IOMatchCategory key in
the drivers’s matching dictionary must be defined.
Unlike the Windows Registry, the I/O Registry is never written to disk or saved between
reboots of the computer.
include the macro OSDeclareDefaultDestructors()
OSSymbol(which is a subclass of OSString). When a new instance of OSSymbol is created, the constructor checks for an existing OSSymbol object that contains the same string value, and if
one is found, returns an instance of the existing object rather than creating a new instance.
All libkern container classes can be iterated over using the class OSCollectionIterator.

chapter 5

Interacting with Drivers from Applications

In the kernel, a driver that provides a serial port will create an instance of the standard
I/O Kit class IOSerialStreamSync. The I/O Kit’s serial family will create a device node in
the /dev directory, publishing the path of the node in the I/O Registry so that applications
can find it.[#define super IOSerialDriverSync] -> IOSerialBSDClient

IOServiceMatching/IOServiceGetMatchingServices/IOIteratorNext

notificationPort = IONotificationPortCreate(kIOMasterPortDefault);
runLoopSource = IONotificationPortGetRunLoopSource(notificationPort);
CFRunLoopAddSource(CFRunLoopGetCurrent(), runLoopSource, kCFRunLoopDefaultMode);

kr = IOServiceAddMatchingNotification(notificationPort, kIOFirstMatchNotification, matchingDict, DeviceAdded, NULL, &iter);
DeviceAdded(NULL, iter);

CFRunLoopRun();

IONotificationPortDestroy(notificationPort);

IOServiceAddInterestNotification

1 2	// kIOGeneralInterest if (messageType == kIOMessageServiceIsTerminated)

Modifying Driver Properties

State-Based Interaction
To establish a connection to a driver, an application simply has to call the function IOServiceOpen(). -> IOUserClient

bool custom_IOUserClient_class::start(IOService *provider)
{
    printf("FlockFlockClient::start client start\n");
    if (! super::start(provider))
        return false;
    
    m_driver = OSDynamicCast(custome_IOService_class_ptr, provider);
    if (!m_driver)
        return false;
    return true;
}

IOConnectCallScalarMethod
IOConnectCallStructMethod
IOConnectCallMethod

Unlike the Windows driver model, which, unless explicitly enabled, does not allow an
application to send multiple control requests to a driver simultaneously, the I/O Kit
allows as many threads to send requests to a user client as the application requires.
Allocating a Port on Which an Application Can Receive Notifications When an Asynchronous
Operation Completes
The I/O Kit framework provides asynchronous variations of each of the IOConnectCallXXX()
functions named IOConnectCallAsyncXXX(). The asynchronoues form of these functions take
additional arguments, including a notification port, a callback function, and a context
parameter that is passed to the callback function.
IOConnectCallAsyncScalarMethod -> sendAsyncResult64
To prevent the user client object from being released while the operation is in progress,
the method increments its retain count when starting the operation and decrements its retain
count when the operatin completes. Finally, when the background operation hash completed, the
user client (or driver) signals the user application by calling sendAsyncResult64().
1
2
3
4
IOReturn
IOUserClient::_sendAsyncResult64 ->
mach_msg_send_from_kernel_with_options/mach_msg_send_from_kernel_proper
-> kernel_mach_msg_send

chapter 6

Memory Mangement

In fact, if you use I/O Kit, it will do all the required translations for you automatically
if you use IOMemoryDescriptor, which is discussed later in this chapter.
All current-generation Macs are litter-endian, as the Intel x86/x86_64 processors are little-endiansl; so too are ARM-based iOS devices. The older PowerPC-based Macs were big-endian.
vm_page_grab/vm_page_alloc
As a kernel programmer, you can create your own zones with the zinit() function if you have
a need for frequent and fast allocation and de-allocation of data objects of the same type. To create a new zonem you need to tell the allocator the size of the object, the maximum size of the queue, and the allocation size, which specifies how mush memory will be added when zone is exhausted.
The kalloc family provides a slightly higher-level interface for fast memory allocation.
The kalloc functions and variants, except kalloc_noblock(), may block (sleep) to obtain memory. The same is true for the kfree() function. Therefore, you must use kalloc_noblock() if you need memory in an interrupt context or while holding a simple lock.

I/O Kit Memory Allocation

IOMalloc/IOMallocAligned/IOMallocPageable
There is also a last variant, IOMallocContiguous(), that allocates memory that is physically contiguous. Its use is now deprecated. Apple recommends using IOBufferMemoryDescriptor instead.

Mapping Memory from a User Space Task into Kernel Space

Note: It is not necessary to map memory into the kernel unless the kernel needs to actively modify it. If DMA is performed from a user space buffer and the data in the buffer does not have to be modified by the kernel, it is not necessary to map it into the kernel’s address space, the buffer can be transferred directly to a hardware device. See Chaptr 9 for more information about DMA.

Mapping Memory form the Kernel to a User Space Task

1	IOConnectMapMemory -> kIOMapAnyWhere

Mapping Memory to a Specific User Space Task

Apple recommends not mapping memory obtained from functions such as IOMalloc() and IOMallocAligned() (though it is possible using the latter) because they come from the zone allocator, which is intended for private and temporary allications and not for sharing. The recommended way of mapping memory is to use the IOBuferMemoryDescriptor, a subclass of IOMemoryDescriptor that also allocates memory.

chapter 7

Synchronization and Threading

Atomic Operations Provides By the Libkern Framework [libkern/OSAtomic.h]
OS[Increment/Decrement/Add/BitAnd/BitOr/BitXor]Atomic
OSCompareAndSwap/OSTestAndSet/OSTestAndClear

An Implementation of Object Reference Counting in a Multithreaded Environment

void Object::retain()
{
	OSIncrementAtomic(&retainCount);
}
void Object::release()
{
	uint32_t originalValue;
	originalValue = OSDecrementAtomic(&retainCount);
	if (originalValue == 1)
		this->free();
}

The compare and swap function can be used to build more complex atomic operations.
Note: All atomic operations, such as OSAddAtomoc(), OSIncrementAtomic(), and OSBitOrAtomic() can be implemented using only OSCompareAndSwap(). In face, a number of atomic functions provided by the libkern library are implemented this way, including all bitwise atomic operations and the 8-bit and 16-bit variations of each operation, which perform a compare and swap on the full 32-bit word containing the value being modified.
Locking
IOSimpleLock -> spin lock
IOLock -> traditional mutex
IORecursiveLock
IORWLock
Unlike a mutex, an IOSimpleLock will never suspend the running thread. Instead it will spin until the lock becomes available. This makes IOSimpleLock perfect for providing synchronization between code that runs within a primiary interrupt handler and non-interrupt code. In reality, this functionality is rarely needed within an I/O Kit driver since most drivers won’t ever have to hanlde an interrupt directly and, if they do, most will defer the interrupt to a secondary intterupt handler. The I/O Kit provides other locking mechanisms that are appropriate for secondary interrupt handles, which are discussed later in this chapter.
Mutex. Because a mutex may block if it cannot be acquired immediately, mutex locks cannot be used within an interrupt handler.
Condition Variables
IOLockSleep/IOLockWakeup
The behavior of IOLockSleep() is similar to that of its equivalent user space function pthread_cond_wait.
Condition variables in the I/O Kit do not have a specific type, rather a condiation variable is an arbitary void* that uniquely identifies an event. A driver will usually use the address of an instance variable (such as the address of the lock itself) as a condition varibale, since the use of an address guarantees that the value will be unique among multiple instances of the driver and other drivers in the system.
Read/Write Mutexes
The I/O Kit provides read/write mutexes through an object known as IORWLock.

GetMachLock
IOSimpleLockGetMachLock -> lck_spin_t
IOLockGetMachLock -> lck_mtx_t
IORecursiveLockGetMachLock -> lck_mtx_t
IORWLockGetMachLock -> lck_rw_t

_pthread_mutex_fairshare_lock_wait ->
__psynch_mutexwait -> ksyn_wait
-> pthread_kern->psynch_wait_update_complete
-> turnstile_update_inheritor_complete
// ksyn_wait type
typedef enum thread_snapshot_wait_flags {
	kThreadWaitNone                 = 0x00,
	kThreadWaitKernelMutex          = 0x01,
	kThreadWaitPortReceive          = 0x02,
	kThreadWaitPortSetReceive       = 0x03,
	kThreadWaitPortSend             = 0x04,
	kThreadWaitPortSendInTransit    = 0x05,
	kThreadWaitSemaphore            = 0x06,
	kThreadWaitKernelRWLockRead     = 0x07,
	kThreadWaitKernelRWLockWrite    = 0x08,
	kThreadWaitKernelRWLockUpgrade  = 0x09,
	kThreadWaitUserLock             = 0x0a,
	kThreadWaitPThreadMutex         = 0x0b,
	kThreadWaitPThreadRWLockRead    = 0x0c,
	kThreadWaitPThreadRWLockWrite   = 0x0d,
	kThreadWaitPThreadCondVar       = 0x0e,
	kThreadWaitParkedWorkQueue      = 0x0f,
	kThreadWaitWorkloopSyncWait     = 0x10,
	kThreadWaitOnProcess            = 0x11,
	kThreadWaitSleepWithInheritor   = 0x12,
	kThreadWaitEventlink            = 0x13,
	kThreadWaitCompressor           = 0x14,
} __attribute__((packed)) block_hint_t;
// pthread_workqueue.cpp
static inline event_t
workq_parked_wait_event(struct uthread *uth)
{
	return (event_t)&uth->uu_workq_stackaddr;
}

static inline void
workq_thread_wakeup(struct uthread *uth)
{
	thread_wakeup_thread(workq_parked_wait_event(uth), get_machthread(uth));
}

thread_wait -> assert_wait -> thread_block
lck_mtx_sleep -> assert_wait/thread_block

Synchronizing Asynchronous Events: Work Loops

To simplify the work required by the driver developer, the I/O Kit provides a class known as IOWorkLoop that creates a single thread on which all asynchronous evetns are handle. In I/O Kit nomenclature, this thread is known as a “work loop” and a driver reigster any of its asynchronous event sources, such as interrupt hanlders and timers, with an IOWorkLoop object.

IOTimerEventSource::timerEventSource
QEventLoop::exec -> QEventLoop::processEvents
QEventLoop::WaitForMoreEvents
mac -> CFRunLoopRunInMode/IOWorkLoop(driver)
win -> MsgWaitForMultipleObjectsEx
unix -> qt_safe_poll [d->pollfds.append(d->threadPipe.prepare());]
glib -> g_main_context_iteration

An driver will typically specify a static method in its main driver class as the callback function of an IOEventSource.

IOCommandGate
A command gate is installed on a work loop like any other event source but, instead of generating events itself, it is used to execute an arbitrary callback function on the work loop thread. [runAction()]
The IOCommandGate achieves its synchronization throught a recursive lock. This allows a function that has been called throught the IOCommanGate to call other functions through the same IOCommandGate without causing a deadlock.
Timers
IOTimerEventSource
Releasing Work Loops
The first part is usually performed by a driver in its stop() method, but the work loop itself is typically not released until the driver’s free() method is called.

Kernel Threads

In these cases, a driver can create its own kernel thread on which to continue executing code without tying up the thread of a user process or the work loop thread.
kernel_thread_start/thread_deallocate/thread_terminate
Note that althought it might seem that it is possible to termiate a background thread from another thread, this will fail with an error returned to the caller. The thread_termiate() function only allows the current thread to ber terminated.

chapter 8

Universal Serial Bus
It is worth mentioning that USE drivers can bewritten both in the kernel and in user space.

I/O Kit USB Support

IOUSEFamily -> com.apple.iokit.IOUSBFamily

chapter 9

PCI Express and Thunderbolt
IOPCIFamily

I/O Kit Interrupt Mechanisms

IOFilterInterruptEventSource/IOInterruptEventSource

Direct Memory Access

chapter 10

Power Management
IOPMPowerState

1 2	// Wake any threads that are blocked on a power state change IOLockWakeup(m_lock, &m_devicePowerState, false);

chapter 11

Serial Port Drivers
It is very flexible interface. The serial driver is concrened only with transferring date bytes between the serial port and the user space application.

Devices that use serial ports include GPS receivers and barcode scanners. They are also commonly used to provide debugging output on hardware.
IOSerialFamily

chapter 12

Audio Driver

chapter 13

Networking

Another feature of the XNU kernel is the network kernel extensions (NKE) mechanism. NKE allows filtrs to be inserted at various levels of the network stack, such as in sockets layer or IP layer. The NKE architecture allows you to write custom routing algorithms, and implment new protocols and virtual network interfaces. It can also be used for packet filtering and logging. Furthermore, the kernel supports the Berkeley Packet Filter (BPF), which allows raw network traffice to be routed to user space for analysis with tools such as tcpdump.

Network Memory Buffers

Network Memory Buffers, or mbufs, is a fundamental data structure in BSD UNIX systems, including Mac OS X and iOS.
mbuf_copydata/mbuf_copyback

Network Kernel Extensions

pcap -> bpf -> nke/ke

chapter 14

Storge Systems
IOBlockStorageDevice

chapter 15

User-Space USB Drivers
Not all hardware devices can be controlled through a user-sapce dirver; for example, devices that contains a memory-mapped address range require a kernel driver. Similarly, devices that generate interrupts need a kernel driver, since only kernel code can execute at primary interrupt level. This means that all PCI and Thunderbolt devices need to be supported by a kernel driver. However, USB- and FireWire- based hardware devices are perfect candidates for a user-space driver.

An application doesn’t need to call the methods from the user client class directly; instead, the I/O Kit framework provides a high level API to control the hardware. This API is known as IOUSELib.
IOCreatePlugInInterfaceForService
Note: If you are familiar with Microsoft’s Component Object Model (COM), you will instantly recognize the method name QueryInterface(). All IOUSBLib classes are based on the COM programming model and are derived from the base class IUnknown. The biggest impact of this design on an application using IOUSBLib is that all IOUSBLib objects are references counted; they can be retained by calling the method AddRef() and can be release by calling the method Release().

chapter 16

Debugging

Common Types of Problems

Race Conditions
Deadlocks
Lock Contention
Access to Invalid Memory
Memroy and Resources Leaks
Illegal Instruction/Operand
Blocking in Primary Interrupt Context
Volunteered Panics

Printing Stack Traces

IOLog/OSReportWithBackTrace()

chapter 17

Advanced Kernel Programming
IOServiceOpen/CFMachPortCreate/IOConnectSetNotificationPort/mach_msg_send_from_kernel

chapter 18

Deployment