The Mantilogs API: Part 4 Serializing and Unit Tests

Welp, it's nearly 8am now, figured I would get started planning before I do anything. Gotta boil some water for a shower this afternoon as well as make more calls so I don't expect to get much done.

To start out I need a way to only return the logs associated with a specific gecko by ID. Then I need to do the thing I dislike, writing tests. This will require a little research as I am not sure if there's anything special required for writing unit tests for Django Rest Framework since it's been a while since I needed to and I haven't done it many times. Gonna have to grab the notebook for this one.

I really don't like the documentation of DRF, I don't know if it's the formatting or what but I have to reread things several times in some cases.

I rarely even write tests for Django, come to think of it. Most of the things I do with it just rely primarily on it's code and there's no point in me testing the libraries, only my own code. Thinking about it I don't really even remember how Django tests work to begin with. Only vague memories of constructing a user and some CRUD operation tests but not how I did them.

Guess I better have a read through of the Django Testing docs as well. What started out as a simple interface redesign for a form turned into quite a journey.

I don't think I need to test my models since all the interaction with them and the heavy lifting will be done by the API but I suppose I should probably try and get as much coverage as possible, you never know... might save me trouble somewhere down the line or just be a waste of time. We'll find out one day.

So to start out I should probably write a helper function to create a test user with staff privileges and another one without. Then I need to test the models, make sure they are requiring the right fields, etc.

Oh boy. Here we go.

What should I even test? I'm not even sure what I would need to test with the models. Anything I can think of testing I would be covering in the API tests. Maybe I should write the API tests first, then come back to the Models.

Have I mentioned I hate writing tests? Every time I have to do it I suddenly lose all energy and just want to go take a nap. Unit tests suck. I know they are important but I still want to skip them every time. Right now I have to fight myself to keep going with writing them. Bah, better get to it.

The amount of foreign keys here is going to be... I hate tests.

I should probably test with tokens rather than session since I plan to use tokens for other software that will interact with the API later on...

It's gonna be one of those days.

So somehow the TEST database that is created when running tests is not getting the correct table structure. This means that the bit I deleted yesterday to fix my migrations was never applied in an earlier migration like I thought. SOMEHOW I managed to migrate the changes to the database at some point and the codebase I am currently working in has no recollection of it.

I suppose now that I have run the migration that threw the error I can re-add the troublesome code and hypothetically fix this error going forward. Guess it wont hurt to try. Good thing I posted the code in question to this blog yesterday or I would be digging through the git repo looking for it.

Well, that worked... thankfully. However a strange new problem has presented itself; following the documentation to the letter has thrown an unexpected error.

AttributeError: type object 'Token' has no attribute 'objects'

token = Token.objects.get(user__username='testadmin')
client = APIClient()
client.credentials(HTTP_AUTHORIZATION='Token ' + token.key)

from rest_framework.authtoken.models import Token
from rest_framework.test import APIClient

# Include an appropriate `Authorization:` header on all requests.
token = Token.objects.get(user__username='lauren')
client = APIClient()
client.credentials(HTTP_AUTHORIZATION='Token ' + token.key)

To the internet!

I expected the common usecase for an API requires a token based auth but maybe that's just my own experience. Some APIs don't even require authentication at all come to think of it. Assuming, making asses of us all since time imemorial.

Solve one, get another.

rest_framework.authtoken.models.Token.DoesNotExist: Token matching query does not exist.

Solve that and get another. Of course. Diggy diggy hole.

django.db.utils.IntegrityError: null value in column "user_id" of relation "authtoken_token" violates not-null constraint

I'm really regretting forcing myself to write the tests and I really am starting to hate the documentation for DRF.

I had to EXPLICITLY create the token.

token = Token.objects.create(user=User.objects.get(username='testadmin'))

Really, really regretting this. Here I thought the token-based auth implementation would have been simple. I could have implemented my own API solution and token system in the time it's taking me to integrate and unravel DRF. Every time I use it I end up frustrated and angry.

Now my test runs but fails, I get a 404. I'm not sure if it's me writing the URL incorrectly or not.

I reformatted the URL and now it's a 403 error. I wasted my morning on this.

So apparently the auth token doesn't work. Hooray. What fun. Yeah, it's nearly lunch time and I am starting to think this whole thing was a waste of time.

Welp, guess I should make lunch and feed my dog. Then I gotta take a shower and make some phone calls. Maybe I will cool off and clear my head by the time I come back to this...

I don't know that I will even need the token auth, I just figured it would be a good idea in the long run. I'll have to power through, I started this and now I have to finish it even if it drives me pants-devouringly mad.

There's lunch done... gotta boil some water, feed my dog, take her out for a walk, take a shower and then do the next thing I am dreading for the day... cold calling landlords.

I suppose I will just work within session auth for now, if I need token-based auth I can deal with it then.

Even with session auth I get a 403. It really is just one of those days. Thought I would try a couple things to figure this out while I waited for the water to boil but it looks like I wont be solving this any time soon.

Even fails with a 403 if I force auth, but the auth test I wrote passes so... I guess it's not an auth issue? I don't know. Very annoying little problem. Every single form of auth ends in a 403 despite the auth test which makes sure the very same account is authenticated without error right before passing.

I'll try testing it in postman and see what comes of it.

Yeah, I went and added the authorization to the header of the request and got a 403. Followed the documentation here as well.

So... now what?

Just took a quick shower, got pretty cold in there. Was snowing this morning. Gonna make some calls once the teeth chattering stops.

This is why I prefer working on games. They are far more complex yet somehow easier for me to understand.

I can use the API fine through the web interface provided by DRF so I have no idea why this is an issue. I should have just stuck with the original plan... ah well, here we are, dealing with the nonsense.

I can fire a GET request and get a list of Geckos but trying to create one gets 403'd.

Ah, there it is.

'DEFAULT_AUTHENTICATION_CLASSES': [ 'rest_framework.authentication.BasicAuthentication', 'rest_framework.authentication.SessionAuthentication', ]

Of course, I didn't see it anywhere in the Token Authentication Documentation. Maybe I missed it somehow.

Now that I got it working with Postman I guess it's time to run the tests again... Oooh, a new error. What did I expect? 400.

I can hit the endpoint the same way with the same data and it works fine, the test however fails with a 400. Man, this is... strange.

Finally figured out how to deconstruct the response. Finally, some data to work from!

{'caretaker': [ErrorDetail(string='Invalid pk "3" - object does not exist.', code='does_not_exist')], 'morphs': [ErrorDetail(string='Invalid pk "4d970264-6ad0-4b76-8524-c9f6df32f9b5" - object does not exist.', code='does_not_exist')]}

Of course, there's no morphs in the test database. Damn it, me. Also of course there's not a third ID in the users when there's only one test user. I got so bogged down with all the other errors that my brain went to mush.

For those trying to unwind the response and tried response.body and got an error, the thing you want is:

response.data

This is how I debugged the thing.

response = client.post('http://localhost:8000/api/leo/geckos/', test_gecko_json, format='json')
        print(response.data)

Anyway, now that I have that knowledge I just need to create a morph with dummy data and set the caretaker to the id of testadmin.

PROGRESS!

from django.test import TestCase
from django.contrib.auth.models import AnonymousUser, User
import uuid, datetime
from rest_framework import status
from rest_framework.authtoken.models import Token
from rest_framework.test import APIClient, APITestCase
from .models import Gecko, Defecation_Log, Feed_Log, Enrichment_Log, Interaction_Log, \
    Incident_Log


client = APIClient()

# Helper functions
def create_test_admin():
    user = User.objects.create_user(username='testadmin', password='12345')
    user.is_staff = True
    user.save()
    return user


# API Tests
# ======================================================
# User Auth Test
class AuthTestCase(APITestCase):
    def setUp(self): 
        create_test_admin()

    def test_token_creation(self):
        response = client.post('/api-token-auth/', {'username':'testadmin', 'password':'12345'},format='json')
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        client.credentials()


class GeckoTestCase(APITestCase):
    def setUp(self): 
        self.testadmin = create_test_admin()

    def test_gecko_creation(self):
        Token.objects.create(user=User.objects.get(username='testadmin'))
        token = Token.objects.get(user__username='testadmin')
        client.credentials(HTTP_AUTHORIZATION='Token ' + token.key)
        print("Token: ", token.key)

        test_gecko_json = {
            "name": "Doberman II",
            "nickname": "Dob II",
            "birth_date": "2020-03-19",
            "gender": "F",
            "personality": "Hardly ever bothered by much, quite curious and highly food focused.",
            "bio": "",
            "caretaker_notes": "",
            "acquired_from": "LLLReptile",
            "acquired_date": "2020-05-19",
            "acquired_price": 28.99,
            "captive_bred": "Yes",
            "breeder_name": "Unknoown",
            "breeder_email": "admin@yarrbeard.com",
            "weight": 93.5,
            "length": 0.0,
            "caretaker": self.testadmin.id,
            "morphs": [

            ]
        }

        response = client.post('/api/leo/geckos/', test_gecko_json, format='json')
        print(response.data)
        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
        self.assertEqual(Gecko.objects.count(), 1)
        self.assertEqual(Gecko.objects.get().name, 'Doberman II')

        client.credentials()

Holy hell, man. Who would have thought just getting the damn thing set up would require that much troubleshooting? Well, I probably learned something during this, though I am still in the battlefield here so I wont be sure what that is for a while.

I should probably create a test class for each log, huh? I don't think data persists between test cases... but I can try and find out. If it doesn't I will need to write a Gecko generator. =] Yay for me.

So a gecko generator it is.

Now, equipped with a crash test gecko I have discovered a bug in my feed log model.

{'feed_not_eaten': [ErrorDetail(string='This list may not be empty.', code='empty')]}

Missed a step there. Updated my model to allow the feed_not_eaten value to be blank and now the tests are passing.

So something good did come of this. Well, not really, I would have discovered that when I was doing my own tests before I built the UI. But hey, maybe it'll be useful one day, right?

Now that's one that I wouldn't have caught. Somehow there's no auth required to add a feed log. One day is today.

So now I have to figure out why I don't need to be authenticated to post to the feed_log endpoint but I do need to be authenticated to post to gecko which is set up the same way as I recall.

Allllrighty then. Now I don't get the 403 when trying to create while unauthorized... probably because I added the basic auth scheme between then and now. That would mean all my endpoints are public right now.

Ah, had to add token auth to the config along with the basic auth.

# Rest Framework Config
REST_FRAMEWORK = {
    # Use Django's standard `django.contrib.auth` permissions,
    # or allow read-only access for unauthenticated users.
    'DEFAULT_PERMISSION_CLASSES': [
         'rest_framework.permissions.IsAuthenticated',
         ],
    'DEFAULT_AUTHENTICATION_CLASSES': [ 
        'rest_framework.authentication.BasicAuthentication', 
        'rest_framework.authentication.SessionAuthentication', 
        'rest_framework.authentication.TokenAuthentication',
        ]

}

That's got it.

Now the tests check to make sure unauthorized users can not post anything to the API and that authorized users can.

It's time for me to clock out for the day. Hopefully tomorrow will go a bit smoother with the rest of the tests.