Programming

May 1, 2024 ∞

Some interesting things we can do with strings in Swift

Today I want to discuss strings in Swift

Hello everyone. I am trying to reinforce my knowledge of some interesting topics by writing about them./ I like to write as this gives me an introspection into my thought process, allows others to get possibly interested in a topic and learn from me, and allows me to learn from much better people than myself. If I have time, I will try to write those articles here. Some of them might be blind-related and others will be about mainstream programming. Keep in mind that I am not a programming God, I might make errors and feel free to correct me.

What are we talking about today?

Everyone, even the most novice programmer is using strings. They allow us to represent text in our programs, and text is something that drives our everyday life in many ways. We read articles, write messages to our loved ones and make programs, of course. So we could think that strings are rather uninteresting, sugarcoated array of characters, for example C-Style strings are just an array of characters. Even if an underlying implementation is somewhat different there’s a pretty easy way to use strings just like a char array. For example in C# we could do something like

string test = "Hello, world!";
char test2 = test[2]; // returns 'l' since that's the 3rd character in our string.

That trick works in many languages, including Python. But in Swift, the language used mostly in the world of Apple we need to give this seemingly simple task a little bit more thought. Let me show you:

var a = "ABCDEF"
var b = Array(a)
print(b[2])

It is still a simple task but as you can see, we first need to explicitly convert our string to an array of characters. By the way I recommend you do one more test for yourself. What result will be returned by the following statement? print(type(of: b[2])) This whole mess is because Swift strings are fully Unicode-aware. If you have played some Chinese, Japanese, Russian ETC games and you had to use Locale Emulator you have seen the lack of proper Unicode support in practice. The problem with Unicode though is that some characters, especially emoji might be composed of more than one Unicode scalars. A scalar is a basic unit in Unicode, for example some emoji have skin tones which are represented by a scalar. A skin tone without a thumbs up, for instance does not make much sense, but if we allowed for using a Swift string like a char array, we could, in theory get an unprintable character that makes no sense. What I want to say here is that because Swift makes some simple things harder for us, we get many more more grant, useful processes simplified since the language does a lot on our behalf. However Swift strings are even more powerful than that, and there are many APIs from Apple that allows us to work with them in a variety of interesting ways. By the way here the border between a SwiftString and NSString blurs. I know that my wording is not entirely correct, but it is correct enough for our purposes. If you want to read more, there are websites such as NSHipster that will explain the nuances much better than I ever could.

Get a name of an Emoji

Have you ever wondered how VoiceOver knows all the names of various Emoji you encounter in your day to day iPhone usage? There’s just a database stored somewhere which has all the names of those, and we can get them in our code as well. Let me show you an example.

var emoji = "☭"
/*1*/let coreFoundationString = NSMutableString(string: emoji) as CFMutableString
/*2*/var range = CFRangeMake(0, CFStringGetLength(coreFoundationString))
/*3*/CFStringTransform(coreFoundationString, &range, kCFStringTransformToUnicodeName, false)
print(coreFoundationString)

This code is pretty long and convoluted as is very much non-swift (Welcome to the world of Apple APIs) but after breaking it down it isn’t that bad. And if you were to do many such operations, you would probably abstract it in a more Swifty way anyway so there’s no problem.

We make an NSMutableString from our swift String and immediately convert it to a CFMutableString type. This is because the methods we use later are the part of CFMutableString type. Don’t worry about any performance implications here, the conversion between the two types is basically free.
We make a CFRange that spans throughout the length of the string. Apple likes to operate on ranges a lot while dealing with strings and you are going to see that a lot if you decide to dig deeper.
Here lies the heart of our quest. The function takes four parameters in this case. The first parameter is the string to be transformed, the second parameter requires a few more words. We pass in the range we created in point 2 but we do something that is very rare in Swift, we pass the pointer to the range, not the range itself, hence the & symbol. The third parameter specifies the actual transformation we desire. In this case, we want to get the Unicode name of the symbol, but there are some more interesting options to choose from, for example, get the latin representation of Cyrillic letters, strip diacritics from strings, ETC. The fourth parameter specifies if we should perform the transformation in reverse (true). I couldn’t find any information online which transformations support reversing, so you need to experiment yourself. The return value is a bool. If it’s true, it means that our transformation succeeded.
We finally print our string. If we see the Xcode’s log we see that we might want to prettify the string a little before using it in production, but the result is OK and it is very easy to make it production ready.

Render HTML in our user interface without `WKWebView`

HTML is very popular when we deal with data from the Internet. Sometimes we need to render it inline in our user interface and we don’t want to spawn a browser for that. Here a type called NSAttributedString come into play. It is a crazily powerful type that is basically a string, but with additional metadata allowing us to perform some advanced operations with relative ease. Let me show you how we can render HTML inline, useful for example when making labels for elements. This code snippets has some SwiftUI and UIKit code in it but we don’t care about it. The reason I included it is completeness.

	var text:  String
	func makeUIView(context: Context) -> UILabel {
		let label = UILabel()
		DispatchQueue.main.async {
			/*1*/ if let data = text.data(using: .unicode) {
				/*2*/ if let attrString = try? NSAttributedString(data: data, options: [.documentType: NSAttributedString.DocumentType.html], documentAttributes: nil) {
					label.attributedText = attrString
				}
			}
		}
	
	return label
	}

The code is long but we have only 2 points of interest.

We make a Data object out of our String type. Data is just a byte buffer in the memory. We need this buffer in order to make a formatted string out of it in step 2.
We make our attributed string. The first parameter is the data object we want to use as an input. The second parameter is an array of options controlling the rendering process. Here we set our document type to be HTML, we can very easily render Markdown, for example. Actually we don’t even need the NSAttributedStringfor that. We can use the third parameter to pass any additional attributes we might want, such as the author of the document. I left it atnil` here. As we can see, a relatively simple code can perform an actually useful operation on the string. Another thing is that it scales well, we could very easily make a simple HTML to markdown converter in literally few lin es of code. Would it be good? No but the point still stands.

Control how VoiceOver reads strings

We are still staying in the world of NSAttributedString. There are some attributes which make it possible to somewhat control how the string is spoken by VoiceOver. Let me show you a very simple example, taken from Orion Gomez’s audiogame utilities for Swift repository. let attributedLabel = NSAttributedString(string: text, attributes: [NSAttributedString.Key.accessibilitySpeechQueueAnnouncement: true]) This is how we make queued announcements. But there are many more accessibility attributes of which some I will discuss below.

accessibilitySpeechPunctuation: Allows to override default punctuation settings of VoiceOver. Useful when we need the user to know the punctuation, for example when reading some kind of code.
accessibilitySpeechPitch: allows to speak a text with a voice that has a higher or lower pitch than our VoiceOver defaults.
accessibilityTextHeadingLevel: allows to set a custom heading level for a text. Useful when presenting raw markdown to a user. It preserves heading navigation around the text.

Epilogue

This is my first serious article about programming. It cost me a lot to write it because I am very stressed about the reception. I hope that at least one person will benefit from it.

Jun 27, 2023 ∞

Modding games for blind gamers: a nice challenge or a bottomless pit of sorrow and frustration??

Introduction

It is widely known that billions of people every day enjoy video games, whether on their computers, mobile phones or gaming consoles. But how many of you thought about making those great titles accessible to those with special needs?

Recently more and more games get accessibility options. And while some games make only the minimum effort to comply with the existing accessibility laws and regulations, some titles take this a step further and make their titles fully playable for disabled people. One of the newest example is a recent remake of the widely awarded “The Last Of Us” game that includes an abundance of various accessibility options which allowed me as a fully blind gamer to complete it autonomously.

However as said before, not all titles get such treatment. But don’t forget that we also have a huge modding scene - countless people dedicating their free time and resources to improve our favorite titles and making them alive for the years to come. Today’s post is dedicated to those modders, as without them, the gaming scene would be dominated by money-hungry corporations of the 21st century. However I digress, and today we have a lot of things to work through, so let’s start right away.

Playable VS. accessible

Those of you who are either disabled themselves, have disabled friends and family or just lurk in the communities for disabled gamers, often hear the terms “accessible” and “playable”. It is widely accepted to use those terms interchangeably, however that is the first mistake which often leads to misunderstandings, false hopes, and wasted time and/or money.

An accessible game, by definition is a game that can be played autonomously by the target audience without the help of external tools and intervention of others. An accessible game in context of a blind gamer means having a full speech support or equal, a way to convey all the game world’s information through speech, sounds or other means. Of course those requirements vary from title to title, as different games have different mechanics and requirements. Let’s take the Rhythm Heaven Fever game for the Nintendo Wii as an example.

The game is a rhythm game, requiring the player to complete various levels in accordance with the music: here we can see right away that music plays a very important role. That, theoretically checks the requirement that says that an accessible game must convey all the necessary information through speech and/or sounds.
The game includes a sound that tells the player when there’s a special challenge available for him/her to complete: that also checks the requirement about making the game’s UI accessible, as the player does not require any external help to get an information about special challenges available to him/her.
The game’s story screens require an OCR technology or a sighted person to read them: this point effectively exempts the game from being called “accessible”, as not all of the information are conveyed to the player.

What is a playable game then? Well, a playable game is a game that can be played and enjoyed by the target audience, however It doesn’t convey all the informations to the player via the native means and/or some features of the game are not available for the target audience at all.For example, the Soul Calibur VI game can be played by a blind person, as the combat generates all the necessary audio output, however actions such as changing the warrior’s weapon or outfit require considerable effort and so this game cannot be called accessible, it is only playable.

When designing your own game mod or an accessibility module for a game you own and control, you need to take those elements into consideration in order to set yourself a set of clear goals and make a plan on how to take on your project. I can tell you that this is the element that is a biggest failure in my mod that I am going to describe below

Designing a mod of your dreams

A word of warning: this, and subsequent sections of this article are going to be highly technical, in that I will illustrate my words with code snippets taken from my game mod. But if you want to follow along, you don’t need to be a programmer. Just listen and read, and it shall be explained to you.

A vision for making my own game mod started somewhere around 2007, 2008 when my elder brother was playing the Gothic II: Night Of The Raven game. As a child I fell in love with the world, characters, music, sound effects and the rest of the game’s lore, but there was a small problem. I am blind, and so playing such a complex 3D game wasn’t possible for me. ANd so I started to search, in my childish stupidity a way to play my favorite game. My brother was no programmer, he knew that there are ways to extend Gothic/Gothic II with additional content, that is monsters, quests, characters or even worlds and he just told me what he knew. Then I have already decided: “I need to be a programmer”.

Years flew by and I already made some serious mistakes when learning programming, the least of those is starting with the C language as my first one. However I was more and more frustrated as I was nowhere close to making a Gothic mod that would make the game playable for me. I had some attempts, some more ambitious than the others but all of them were a failure because there were no correct tools for such an undertaking. And here we come to the first conclusion: when starting a game mod, before you rush to write your first line of code learn about all the tools the game offers you, because without proper tooling, even the most brilliant of ideas will fail.

Another few years have gone by by and the Gothic modding scene has seen countless changes, such as release of the Ikarus script pack, Lego or even complete engine reimplementations such as Open Gothic (its still coming on nicely!) or ReGoth. However what changed everything for me was the release of the Union SDK, which, in very simple terms allowed me to edit whatever I wanted in the game Engine. This is when the YAGA project was born…

AFter I have written some basic stuff such as Screenreader abstraction library I have decided to take a little while, sit back and think about how my mod should work from the player’s perspective.

Gothic is a 3D game that is played from the third-person perspective. This is not only about the graphic rendering but also the audio. This is because the game’s audio is played from the camera’s perspective. So a very important thing to do was to force the game to be first-person. I still don’t know how does it all work for sighted people, but here goes the rough implementation. If you are not interested in the programming details, feel free to skip this part. void BlindCameraLoop() { zCVob* cameraVob = ogame->GetCameraVob(); cameraVob->SetSleeping( True ); zVEC3 playerHeadPosition = player->GetPositionWorld() + zVEC3( 0.0f, 70.0f, 0.0f ); zVEC3 playerAtVector = player->GetAtVectorWorld(); cameraVob->SetPositionWorld( playerHeadPosition ); cameraVob->SetHeadingYWorld( playerHeadPosition + playerAtVector ); } }

There are some important parts to this code:

The camera is a vob. Basically everything in the Gothic’s world is a vob which is just an object which has some properties.
At the very beginning of our loop we make the camera sleep to avoid any unnecessary movements which would disturb the player’s hearing.
We get the rough position of the player’s head by getting his position in world coordinates and adding a new vector to it.
We get the player’s heading.
Finally in the last two lines, we fix the camera’s position to the player’s eyes.
Since this runs in a loop, this code is executed basically every frame.

That’s a very important think down, but… its till so little! How the player can navigate around or access the information such as menus, health, journal, ETC? Well, if that is something that would interest you then I have a good news: I am already starting to create the second part of this series, as I want to give people less information so they can learn and comprehend better.