This is entirely why I was somehow (vainly?) hoping for a magic wand solution that would at least reliably generate a figure for the number of valid glyphs in each set. The 8×6 set is more or less the systematic combinations of the 4×6 set with itself, for example - except there are always some additional new possibilities when the glyph size is increased. I look forward to seeing the result once the run is complete. [EDIT: I might use the quarter glyph approach to take a look at the possibilities for the 8×8 set next.]

Great, also, to hear that you're looking into GAP; I had to read through a few forum posts to work out what was going on as well. I've been afk/afc for a few days with sick kiddies, nice weather and some muscular pains that needed dealing with. It will be a matter of me actually scheduling in specific work time for this project from now on (all told, I could probably do with some serious time-management tips and training). Never a dull moment round these parts!

And I'm really glad that you've enjoyed the inspiration on this one - do you want to make a specific appointment to go through the goals as I've envisaged them and compare them with what you feel prepared to do? It sounds like we'll at least get a nice little widget that can perhaps spew out multicoloured glyphs in response to music or any other data stream, with a few knobs and sliders to tweak its responses.

Thanks again for all the work and processor time

Hey,

I thought this could be of interest to you:

https://museumofwitchcra...object/enochian-tablets/

Thanks for your contribution! I could imagine an esotericist having quite a bit of fun with the various sets of glyphs. The already nearly mind-boggling number of possibilities with the 8×8 set would have put that, and particularly anything larger, out of the reach of practitioners of that era - at least in terms of defining the complete set. Finding 149 of them by hand in the 6×6 set was hard enough as it is. It would have required several hours of focused, methodical and dedicated work on top of a certain amount of clear-headed planning to have come up with a succinct and accurate answer in a short time without the aid of electronics.

This question has turned out to be quite a tool for mind development, feel free to have a play with it - I shall be sharing my notes once I get around to scanning them into the computer.

Yes and no. It took a bunch of mathematicians to decipher some of the more versatile tablets of renaissance alchemy and their algorithms. It is fascinating to see what was already happening before, in the case of the i ching even thousands of years.

https://archive.org/details/jimreedssoyga

Yes i think we should have a talk about where to take this and how. Only question is, when For me, any evening around 21:00 would make sense. Or we could schedule something for one of the weekend days at an earlier time.

I improved the brute force a bit, it ran 7x6 in ca. 1h now vs the 5h it took before. With the improvement, the last bit of 8x6 finished yesterday (I hope i didn't skip anything with all the restarting and debugging, gonna have to double check). Here's the summary:

That is indeed a good point. The human mind is a remarkable thing, and back then there was, arguably, much less in the way of distractions.

Given that we're poised to take on the daunting task of examining the enormous set of 8×8 glyphs, I'm feeling a certain resonance with the I-Ching at the moment. Time for a reading, perhaps

HT - 21.00 CEST is good for me most days except this Friday (unless something unexpected shows up). We could simply have a chat room for it and once we've got a specific day pinned down I'm happy to invite other interested parties as well.

Now that you've produced a figure for the 8×6 set I'll attempt to put all these values into some kind of chart or graph. We've got to have at least one graph out of all of this

Meanwhile, I've been sketching a few of the 8×8 glyphs - about 450 of them so far. Only a million or so to go...

The 8×8 set is fascinating and compelling, with so much more scope than the 6×6 set. I've developed a moderately straightforward technique for copying my hand-drawn glyphs into digital versions with clean lines. It's pretty easy to scan the hand-drawn pages into the computer, then use a transparent layer in GIMP to trace over it courtesy of the blank template I've made.

[EDIT: I would imagine it would be possible to automate this tracing process - but the glyph generation code kind of obviates that.]

Attached are a few examples.

There are some things mentioned about code optimisation (particularly Knuth's algorithm X) in this video that may perhaps be applicable to the glyph generation question:

Hehe, that was a fun episode!

I know python isn't the right choice for super fast code, of course. For the original problem of 6x6, that wasn't even relevant. Now that i'm running larger grids, i've become painfully aware of the need for more optimization. I have improved the speed of my code by a few orders of magnitude so far, and i think i can do another 2-3 before i'd have to switch out the fundamental design or even the language.

I can describe the evolution of my code so far, maybe that's fun. (Sorry btw for not posting it yet, but i have more improvements i want to make first).

At first, i created a model that can represent these things. I wrote a few classes:
- Vector, a class that has x and y values as well as a few methods that enable some arithmetic and such.
- Square, represents one square of the grid. They have a position (Vector), an incoming and an outgoing direction, as well as several convenience methods to traverse the grid and measure distances.
- Glyph, has Squares and a bunch of methods for connecting them, rendering as ASCII, SVG, getting rotated or mirrored versions, etc.

I'm not gonna go into my original number-based approach here. I intend to go back to it as soon as i run out of immediate ideas for improving the brute force, and i'll make a post about it if/when i get it to work correctly. I'll only describe my brute force journey today.

The brute force per se is very straight forward. It is a recursive function, calling itself to a depth of x*y (space size), each time appending one step that can be left, right, or forward.

In a 6x6 grid, the upper bound is 3^36 or 150094635296999121 different sequences of moves. Let's call them leaves of the tree. If i can check a million per second of them for validity (and i can't with this design), that's about 4759 years. Luckily, we don't have to generate or try them all. We can skip branches when we discover that they can't lead to any valid leaves.

My first naive approach was to detect that condition only when it's too late. That is, when i try to connect to a square that's outside the grid, or to a square that's already connected. This way, there was still dozens to hundreds of million sequences to try (but, about 10 orders of magnitude less than the worst case). It took a few hours iirc.

I then added another detection of dead branches: In the field right before the corners, it has to connect to the corner. Going any other direction can never produce a valid glyph. Then in the corner, it has to always turn right. I think that was all i did to get 6x6 down to 15 minutes.

Next thing i did was to detect when i'm stepping on an edge that i have to keep free. For example, when i'm finding a path from the top left to the top right corner, i can never go all the way to the left, right, or bottom edge, or i won't be able to get back to the top left corner later. This was the one that brought 7x6 from over 5h down to about 1h.

At that point, it still got stuck for minutes to hours sometimes, and i didn't immediately realize why. So i added a command to see what it's currently trying. I found that (of course) it sometimes encloses empty squares that are then unreachable.

So next thing i did, was to check for that whenever it returns back to the one legal edge, or touches another portion of the path from a different edge. Also aborting a branch when it leaves just one space to the "home" edge. This brought 7x6 down to 30 minutes.

Next thing i will do is the same detection, but when it touches the same portion of itself. It's more complicated because depending on how many turns it made, i have to check for empty squares on the left or the right side of all the previous squares of that portion. I found the correct way to know which one when the contact is frontal (7x6 now 15min), but haven't had the time yet to extend it for tangential contact. I hope to do that tomorrow. I'll also have to check for single square distances in other places than the home edge.

Running 8x8, after finding about 175k glyphs, i ran into memory problems. This thing was using gigabytes! Up until that point, i kept all the Glyph objects in lists, along with all the Square objects they contain, etc. I realised i can keep only their Lfr string representation and save about 90% of that memory.

During that change, i saw that i can improve the Lfr format by representing the corners by a '.' character rather than an 'r' character. That way, it's immediately obvious where they are, and this change allowed me to rewrite the rotation and mirroring functions to be a few string operations rather than many object manipulations. 7x6 runs in 11 minutes now.

So yeah i'm not done with optimising my inherently non-optimal design yet. Of course some sort of bitmap would be quicker - probably even within python, and certainly in a language closer to the hardware such as C. I don't really feel like rewriting it in such a way though, at this time. I wanna keep improving the python code until i run out of ideas, then implement the necessary bits in javascript to generate and script SVGs.

I've completed all the spaces up to 8x7, 9x6 and 10x5. I'm not gonna upload individual SVG files for them to my server because it's hundreds of thousands of files - i'll change that page so it loads entire spaces per file, then generates the SVGs inside the browser using javasript. This will take a little time though. I hope i can be back at the original number based approach for that, it would make the file sizes much smaller. But if i can't make that work right, the Lfr format will do - it's 20MB for 8x7, less if only including the archetypes and omitting the information about self symmetries. It can be reduced further by gzipping or similar.

Currently i'm running 8x8 (at 570k glyphs so far) and 10x6 (450k glyphs so far). I won't run spaces bigger than that unless i can speed things up by at least another 2 orders of magnitude.

Great to know you're still on this. I've been attempting to classify all the 8×8 quarter-glyphs by hand (I've drawn about 365 of them so far!)

Another conceptual approach I intend to implement somehow is considering how the loop as a whole gets manipulated to fill the glyph space in a valid way. But that idea barely qualifies as embryonic so far. I may just make it out of twigs and string as a kind of board game.

And of course, having the goal of getting to grips with animation naturally meant I've ended up doing some copper working instead and I now have a gigantic cold-finger condenser for some winter... shenanigans. Welcome to my bizarre world of Pathological Demand Avoidance!

Oh yeah i'm also very good at avoiding what i should do. One could say, this project is for me a bit like the copper condenser is for you

I just did another major speed increase. And so easily. Crazy that i didn't think of this one sooner. I take the last connected square, go to the previous square from that one, and check all it's neighbors if they have more than 2 occupied direct neighbors. If so, that neighbor is unreachable, and the current branch can be skipped.

6x6 takes 44 seconds now, 7x6 ca. 5 minutes, 8x6 under an hour. The bigger spaces seem to never get stuck for more than a few seconds now.

Note that all the times i mention are always including categorisation, ie. finding out if each glyph is archetypal, or what transform of which archetype it is. This categorisation currently takes a lot of time, growing with the number of already computed glyphs. And i just had an idea how to get that time to be nearly constant within a space of given size, regardless how many glyphs are already known. Gonna do that now.

Times are also including a ton of console output, so i see what it's doing. Omitting that would save a bit, but imo not enough to justify having no output until it's done.

C++ is going to be considerably faster than python.

I assume the function calls itself 4 times, one per direction?

Also, anything that cuts the plane into two halves and then goes back into the direction it came from on the opposite wall is an early out, except on the edges.
This could be extended to some sort of map that checks for unreachable spots or "bubbles" everytime the possibility of creating those bubbles happens.
Though this might cost more than just leaving them and traversing the invalid glyph.

I think that makes sense anyways.

Yeah C++ would be a lot faster.
I don't have the same degree of fun (or skill) doing it though, and that's my only real motivation here


It calls itself for each square (ie. to a depth of 36 in a 6x6 grid), and tries out the 3 different moves for that square in a loop. Something like this pseudocode:



This was well worth it. It reduced 7x6 from 5min to 3.5, and it's a change that has bigger savings for bigger spaces.

I see.

In case you want to explore this, take the boundary where a new bubble might have formed, take the two cells at both sides of the boundary, now trace from one of them along the faces of the glyph, if it ends up back into itself its and early out.
You could perhaps propagate a sort of signal from one or both cells... etc

Im sure there is an efficient way to detect closed loops, speciallly if the map is being generated one step at a time.
Generating a map storing a gradient of connectedness or something similar.

I guess this this might make more sense the bigger the grid gets.

I'm not yet sure that this is exactly right, i had a little hickup and am gonna run that again, just to be sure. Just thought it was nice that it completed now.

8x8 is well over a million now too, but still has 2 forward steps at the start - it's gonna end up being 2-3M glyphs i think.

Interesting that 8×8 is looking like being such a vast set.

I'm still wondering if there is a way of turning any data input into a valid glyph without referring to a library though.
I do wonder if the code behind the cool plugin in this video has any connection with that aim:


That's complete guesswork on my part - I've not looked 'under the bonnet [hood]' of anything like this and have singularly failed to develop any additional coding skills since the inception of this thread. However, there are things about how that plugin shapes sound that somehow give me the feeling that there could be something in common with my goal in there.

Re-run result came in a few days ago Looks like the first run, being split into parts with code changes, manual interventions, and a change of notation format for the glyphs, missed a few glyphs and missed a few congruences of transforms. I'll have to check whether there's any glyphs found in the first run that weren't found in the rerun (which would prove that the current state of code is incorrect) - i think there shouldn't be, given that the current code found all the same glyphs for all the smaller spaces as did the slower but simpler (and therefore more likely correct) code of their original runs. So i assume that the first runs of 10x6 and 8x8 simply found a subset of what the reruns are finding.

For anyone interested why a bug could lead to skipped glyphs without me knowing, it's because i used AttributeException to detect when a square was being connected to no square (outside the grid). I must've made a mistake at one point that led to AttributeExceptions being thrown there under different circumstances, which went by unnoticed for a while and led to the corresponding branches being skipped.


This is the result of the first run of 8x8, finally done. I can already tell it's much too low though. When the 10x6 result was different, i immediately started a rerun of 8x8 as well. That is currently at 2.8-ish millions, and it's quite far behind in comparison:

The way these numbers have developed in the other runs, i think final result will be well over 5M glyphs.


Well my original number based code can do that. It just can't produce all possible glyphs (yet). I'm now working on a little interactive console UI so i can share it and there's something to play with.

Not quite 5M, but pretty close

Over half a million archetypes, that's crazy!

Great! That finalises the number of possible closed loops on a chess board, starting atone specific square, to put it in maybe more relatable terms. It seems, too, that this means the total number of closed loop tours starting on any square would simply be 64 times the glyph number: 64*4638576 = 296868864. This number may be checkable on a chess website - in fact, does Wolfram Alpha have something like this, or maybe OEIS?

[Our example doesn't correspond to any conventional chess piece, however - and I've just had the horrifying thought of how it would work out for a king's tour incorporating diagonals. Best keep that for smaller boards than 8×8, especially if we start to consider things like queen's tours and the logical conclusion of different ways of hopping from any one square to any other square until all the squares have been visited will obviously give the factorial of (one less than? - if the loop isn't closed) the number of squares on the board. So let's not bother calculating that for the chessboard ]

I mentioned your 8×8 result to my kid and of course was immediately asked what would happen if you used a cube, which is a great question. If you ever got short of ideas, you could adapt your code to trace a loop going through each of the subdivided cubes of a cube or cuboid, via the faces only of course as per the rules. The same stipulation of requiring an even number of subdivisions will apply so cubes would only need to be of even numbers, and cuboids I haven't checked thoroughly but I suspect that having one side of even length would suffice.

It looks as though the complete set of even the archetypal 8×8 glyphs would make for something of an unwieldy card deck, but maybe - just maybe - they could become a trading card series. How are your marketing skills?